Open-field handheld flourescence imaging systems and methods

ABSTRACT

An imaging device having an imaging field of view, the imaging device including at least one illumination port configured to output light for illuminating a target; an imaging sensor to detect light traveling along an optical path to the imaging sensor; and a first movable window positioned upstream of the sensor with respect to a direction of travel of light along the optical path, wherein the first movable window is configured to move into the optical path in a deployed position for modifying light received from the target.

REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional Application Ser.No. 62/457,690 filed Feb. 10, 2017, titled “OPEN-FIELD HANDHELDFLUORESCENCE IMAGING SYSTEMS AND METHODS,” which is hereby incorporatedby reference in its entirety.

FIELD OF THE INVENTION

The present disclosure relates generally to medical illumination andimaging. More specifically, the disclosure relates to illumination andimaging of a target material.

BACKGROUND OF THE INVENTION

Illumination is an important component of imaging systems such as, forexample, broadband imaging systems with self-contained illumination. Inmany applications of imaging systems, such as in medical imaging andespecially in fluorescence medical imaging, it may be challenging toachieve even, full field illumination of the imaging field of view, andalso to provide a sufficient intensity of illumination to yield asufficiently strong imaging signal. Matching the illumination profile tothe imaging field of view is one method of conserving illuminationpower, while multiple illumination ports may be used to provide evenillumination across the field of view. Existing illumination projectionin imaging systems may feature anamorphic projection to match theimaging field of view, but typically only feature a single illuminationport and are not configured for close working distances. Single portillumination systems result in substantial shadowed regions obscuringvision when illuminating complex topography such as, for example, humananatomical structures or other biological materials. Existing designsfor open field surgical imaging and illumination devices may make use ofmultiple illumination ports to minimize shadowed regions, such as a ringlight surrounding the imaging optics, but these designs waste excessillumination that falls outside of the field of view and fail to achieveeven illumination of the field of view over a range of workingdistances.

SUMMARY OF THE INVENTION

According to some embodiments, an imaging device having an imaging fieldof view may include at least one illumination port configured to outputlight for illuminating a target; an imaging sensor to detect lighttraveling along an optical path to the imaging sensor; and a firstmovable window positioned upstream of the sensor with respect to adirection of travel of light along the optical path, wherein the firstmovable window is configured to move into the optical path in a deployedposition for modifying light received from the target.

In any of these embodiments, the first movable window may be configuredto rotate into the optical path in a deployed position.

In any of these embodiments, the first movable window may be configuredto translate into the optical path in a deployed position.

In any of these embodiments, the first movable window may extendperpendicularly to an optical axis in the deployed position.

In any of these embodiments, the first movable window may be configuredto pivot into the optical path in a deployed position.

In any of these embodiments, the first movable window may be configuredto pivot about a first pivot axis extending perpendicularly to anoptical axis.

In any of these embodiments, the first movable window may include afilter.

In any of these embodiments, the filter may be configured to filter outvisible light.

In any of these embodiments, a second movable window may be positionedupstream of the imaging sensor with respect to the direction of travelof light along the optical path, wherein the second movable window isconfigured to move into the optical path in a deployed position formodifying light received from the target.

In any of these embodiments, the second movable window may be configuredto pivot about a second pivot axis extending perpendicularly to anoptical axis.

In any of these embodiments, the first movable window may be configuredto pivot about a first pivot axis extending perpendicularly to theoptical axis and the first pivot axis and the second pivot axis may becoplanar with a plane extending perpendicularly to the optical axis.

In any of these embodiments, the first movable window and the secondmovable window may be coupled to a linkage that is configured tosimultaneously move the first and second pivoting windows.

In any of these embodiments, when the first movable window is in thedeployed position, the second movable window may be moved out of theoptical path in a stowed position.

In any of these embodiments, the image sensor may be translatable withrespect to the first movable window.

In any of these embodiments, the first movable window may extendperpendicularly to an optical axis in the deployed position and theimage sensor may be translatable along the optical axis.

Any of these embodiments may include a first illumination port and asecond illumination port, wherein the first illumination port isconfigured to generate a first illumination distribution at the target,the second illumination port is configured to generate a secondillumination distribution at the target, the second illumination port isspaced apart from the first illumination port, the first and secondillumination distributions are simultaneously provided to the target andoverlap at the target, and the illumination from the first and secondports is matched to a same aspect ratio and field of view coverage asthe imaging field of view.

In any of these embodiments, the first and second illumination ports maybe fixed with respect to each other.

In any of these embodiments, the at least one illumination port may beconfigured to output visible light and/or excitation light.

In any of these embodiments, the image sensor may be a single sensorthat is configured to detect light from the target resulting fromillumination by visible light and excitation light.

In any of these embodiments, the image sensor may comprise separatesensors configured to detect light from the target resulting fromillumination by visible light separately from that resulting fromillumination by excitation light.

Any of these embodiments may include a wavelength-dependent apertureupstream of the image sensor, wherein the wavelength-dependent apertureis configured to block visible light outside a central region.

Any of these embodiments may include one or more sensors for sensing anamount of light incident on the device.

Any of these embodiments may include a control system configured toadjust at least one image acquisition parameter based on output from theone or more sensors.

In any of these embodiments, the at leak one image acquisition parametermay include an exposure duration, excitation illumination duration,excitation illumination power, or imaging sensor gain.

In any of these embodiments, at least one of the one or more sensors maybe configured to sense visible light and near infrared light.

In any of these embodiments, at least one of the one or more sensors maybe configured to sense near infrared light.

Any of these embodiments may include one or more drape sensorsconfigured to detect a drape mounted to the device.

Any of these embodiments may include one or more light emitters foremitting light for detection by the one or more drape sensors.

In any of these embodiments, the one or more drape sensors may beconfigured to detect light emitted from the one or more light emittersafter reflection of the emitted light off of one or more reflectors onthe drape.

In any of these embodiments, the one or more reflectors may include aprism.

According to some embodiments, an imaging system may include an imagingdevice according to any one of the above embodiments, an illuminationsource for providing illumination to the imaging device, and a processorassembly for receiving imaging data generated by the imaging device.

According to some embodiments, a method for imaging a target may includeilluminating the target with an illuminator of an imaging device;receiving light from the target at an imaging sensor of the imagingdevice in a first imaging mode, wherein at least some of the lightreceived at the imaging sensor in the first imaging mode compriseswavelengths in a first band; switching to a second imaging mode; andwhile in the second imaging mode: blocking light of wavelengths outsideof a second band received from the target from reaching the imagingsensor using a first movable filter of the imaging device, wherein atleast some of the blocked light comprises wavelengths in the first band,and receiving light of wavelengths within the second band received fromthe target on the imaging sensor.

In any of these embodiments, the second band may include near infraredwavelengths.

In any of these embodiments, the first band may include visible lightwavelengths.

In any of these embodiments, the method may include, while in the secondimaging mode, sensing light levels at one or more light level sensors ofthe imaging device and adjusting one or more of image sensor signalgain, illumination pulse duration, image sensor exposure, andillumination power based on output of the one or more light levelsensors.

In any of these embodiments, the method may include, while in the firstimaging mode, sensing light levels at one or more light level sensors ofthe imaging device and adjusting one or more of image sensor signalgain, illumination pulse duration, image sensor exposure, andillumination power based on output of the one or more light levelsensors.

In any of these embodiments, switching to the second imaging mode mayinclude moving the first movable filter into an optical path along whichlight from the target travels to the imaging sensor.

In any of these embodiments, switching to the second imaging mode mayinclude moving a clear window out of the optical path.

In any of these embodiments, switching to the second imaging mode mayinclude moving a second movable filter out of the optical path.

In any of these embodiments, the first imaging mode may be switched tothe second imaging mode in response to a user request.

In any of these embodiments, the user request may include a user inputto the imaging device.

In any of these embodiments, the method may include, while in the secondimaging mode, receiving a request from the user to switch to the firstimaging mode; and in response to receiving the request from the user toswitch to the first imaging mode, moving the movable filter out of theoptical path.

In any of these embodiments, the method may include, while in the secondimaging mode, sensing light levels at one or more light level sensors ofthe imaging device and adjusting one or more of image sensor signalgain, illumination pulse duration, image sensor exposure, andillumination power based on output of the one or more light levelsensors; and in response to receiving the request from the user toswitch to the first imaging mode, ceasing to adjust one or more of imagesensor signal gain, illumination pulse duration, image sensor exposure,and illumination power based on output of the one or more light levelsensors.

In any of these embodiments, the method may include detecting an objectat least partially blocking an illumination beam of the illuminator, andin response to detecting the object, adjusting an illumination power ofthe illuminator.

According to some embodiments, a kit for imaging an object may include afluorescence imaging agent and the device of any one of the aboveembodiments or the system of any one of the above embodiments.

According to some embodiments, a fluorescence imaging agent may includea fluorescence imaging agent for use with the device of any one of theabove embodiments, the system of any one of the above embodiments, themethod of any one of the above embodiments, or the kit of any one of theabove embodiments.

In any of these embodiments, imaging an object may include imaging anobject during blood flow imaging, tissue perfusion imaging, lymphaticimaging, or a combination thereof.

In any of these embodiments, blood flow imaging, tissue perfusionimaging, and/or lymphatic imaging may include blood flow imaging, tissueperfusion imaging, and/or lymphatic imaging during an invasive surgicalprocedure, a minimally invasive surgical procedure, or during anon-invasive surgical procedure.

In any of these embodiments, the invasive surgical procedure may includea cardiac-related surgical procedure or a reconstructive surgicalprocedure.

In any of these embodiments, the cardiac-related surgical procedure mayinclude a cardiac coronary artery bypass graft (CABG) procedure.

In any of these embodiments, the CABG procedure may include on pump oroff pump.

In any of these embodiments, the non-invasive surgical procedure mayinclude a wound care procedure.

In any of these embodiments, the lymphatic imaging may includeidentification of a lymph node, lymph node drainage, lymphatic mapping,or a combination thereof.

In any of these embodiments, the lymphatic imaging may relate to thefemale reproductive system.

According to some embodiments, a system for imaging a target includesone or more processors; memory; and one or more programs, wherein theone or more programs are stored in the memory and configured to beexecuted by the one or more processors, the one or more programsincluding instructions for, within a period: activating an excitationlight source to generate an excitation pulse to illuminate the target;receiving an ambient light intensity signal from a sensor during aportion of the period in which the excitation light source is notactivated; exposing an image sensor for a fluorescent exposure timeduring the excitation pulse; receiving outputs from the image sensor;compensating for ambient light based on the ambient light intensitysignal; and storing a resultant image in the memory.

In any of these embodiments, the one or more programs may includeinstructions for, within the period: activating a white light source togenerate a white light pulse to illuminate the target such that thewhite light pulse does not overlap the excitation pulse; and exposingthe image sensor for a visible exposure time during at least one whitelight pulse.

In any of these embodiments, the one or more programs may includeinstructions for exposing the image sensor for a background exposuretime when the target is not illuminated.

In any of these embodiments, the one or more programs may includeinstructions for detecting a periodic frequency of the ambient lightintensity.

In any of these embodiments, compensating for ambient light may includesetting an image acquisition frame rate equal to a multiple or a factorof the periodic frequency prior to exposing the image sensor for thebackground exposure time and prior to exposing the image sensor for thefluorescent exposure time during the excitation pulse; and subtractingimage sensor output received for the background exposure time from theimage sensor output received for the fluorescence exposure time to formthe resultant image.

In any of these embodiments, compensating for ambient light may includesynthesizing or extracting, from one or more received ambient lightintensity signals, a complete periodic cycle of ambient light intensityhaving the detected periodic frequency; extending the ambient lightintensity periodic cycle to a time period corresponding to thefluorescence exposure time; calculating a first accumulated ambientlight value corresponding to an area under the curve of ambient lightintensity during a background exposure time; calculating a secondaccumulated ambient light value corresponding to an area under the curveof the ambient light intensity during the fluorescence exposure time;scaling the received image sensor output for the background exposuretime and the received image sensor output for the fluorescence exposuretime based on a ratio of the first and second accumulated ambient lightvalues; and subtracting the scaled image sensor output for thebackground exposure time from the scaled image sensor output for thefluorescence exposure time to form the resultant image.

In any of these embodiments, the one or more programs may includeinstructions for receiving an ambient light intensity signal from thesensor during the background exposure time.

In any of these embodiments, the one or more programs may includeinstructions for extending the ambient light intensity periodic cycle tothe time period corresponding to the fluorescence exposure time.

According to some embodiments, a method for imaging a target includes,at a system having one or more processors and memory, activating anexcitation light source to generate an excitation pulse to illuminatethe target; receiving an ambient light intensity signal from a sensorduring a portion of the period in which the excitation light source isnot activated; exposing an image sensor for a fluorescent exposure timeduring the excitation pulse; receiving outputs from the image sensor;compensating for ambient light based on the ambient light intensitysignal; and storing a resultant image in the memory.

In any of these embodiments, the method may include, within the period,activating a white light source to generate a white light pulse toilluminate the target such that the white light pulse does not overlapthe excitation pulse; and exposing the image sensor for a visibleexposure time during at least one white light pulse.

In any of these embodiments, the method may include exposing the imagesensor for a background exposure time when the target is notilluminated.

In any of these embodiments, the method may include detecting a periodicfrequency of the ambient light intensity.

In any of these embodiments, compensating for ambient light may includesetting an image acquisition frame rate equal to a multiple or a factorof the periodic frequency prior to exposing the image sensor for thebackground exposure time and prior to exposing the image sensor for thefluorescent exposure time during the excitation pulse; and subtractingimage sensor output received for the background exposure time from theimage sensor output received for the fluorescence exposure time to formthe resultant image.

In any of these embodiments, compensating for ambient light may includesynthesizing or extracting, from one or more received ambient lightintensity signals, a complete periodic cycle of ambient light intensityhaving the detected periodic frequency; extending the ambient lightintensity periodic cycle to a time period corresponding to thefluorescence exposure time; calculating a first accumulated ambientlight value corresponding to an area under the curve of ambient lightintensity during a background exposure time; calculating a secondaccumulated ambient light value corresponding to an area under the curveof the ambient light intensity during the fluorescence exposure time;scaling the received image sensor output for the background exposuretime and the received image sensor output for the fluorescence exposuretime based on a ratio of the first and second accumulated ambient lightvalues; and subtracting the scaled image sensor output for thebackground exposure time from the scaled image sensor output for thefluorescence exposure time to form the resultant image.

In any of these embodiments, the method may include receiving an ambientlight intensity signal from the sensor during the background exposuretime.

In any of these embodiments, the method may include extending theambient light intensity periodic cycle to the time period correspondingto the fluorescence exposure time.

BRIEF DESCRIPTION OF THE DRAWINGS

The patent or application file contains at least one drawing executed incolor. Copies of this patent or patent application publication withcolor drawing(s) will be provided by the Office upon request and paymentof the necessary fee.

Features will become apparent to those of ordinary skill in the art bydescribing in detail exemplary embodiments with reference to theattached drawings in which:

FIG. 1 illustrates a schematic view of a system for illumination andimaging according to an embodiment;

FIG. 2 illustrates a schematic view of an illumination module accordingto an embodiment;

FIGS. 3A and 3B illustrate a schematic side view and plan view,respectively, of an exemplary lens module in a steerable housingaccording to an embodiment;

FIG. 4A illustrates a schematic view of a linkage for synchronousfocusing of the imaging system and steering of the illumination systemaccording to embodiments;

FIGS. 4B and 4C illustrate a bottom view and a top view, respectively,of a linkage for synchronous focusing of the imaging system and steeringof the illumination system according to embodiments;

FIGS. 5A and 5B illustrate bottom views of the linkage at a far workingdistance and a near working distance, respectively, according to anembodiment;

FIGS. 6A and 6B illustrate a perspective top view and a perspectivebottom view of an illumination and imaging system according to anembodiment;

FIG. 7 illustrates an enclosure according to an embodiment;

FIGS. 8A and 8B illustrate perspective views of different exemplarypositions in which the system may be used;

FIG. 9A illustrates a drape for use with the system according to anembodiment; FIGS. 9B to 9E illustrate perspective, front, top, and sideviews, respectively, of a drape lens and frame for use with the systemaccording to an embodiment; FIG. 9F illustrates a drape lens and frameinstalled on an enclosure of the system, according to an embodiment;FIG. 9G illustrates a section view of the installed drape lens and frameon the enclosure of the system of FIG. 9F;

FIGS. 10A to 10D illustrate illumination distributions for differentillumination configurations;

FIG. 11A illustrates a timing diagram for visible and excitationillumination and image sensor exposures according to an embodiment;

FIG. 11B illustrates a timing diagram for visible and excitationillumination and image sensor exposures according to an embodiment;

FIG. 11C illustrates a timing diagram for visible and excitationillumination and image sensor exposures according to an embodiment;

FIG. 11D illustrates a timing diagram for visible and excitationillumination and image sensor exposures according to an embodiment;

FIG. 11E illustrates a timing diagram for visible and excitationillumination, image sensor exposures and ambient light measurementaccording to an embodiment;

FIGS. 12A to 12C illustrate pixel layout and an interpolation schemeaccording to an embodiment;

FIGS. 13A to 13C illustrate diagrams of an embodiment of a displaymethod output when a target reticle is placed over regions with nofluorescence intensity, high relative normalized fluorescence intensity,and moderate relative normalized fluorescence intensity, respectively;

FIG. 13D illustrates a diagram of an embodiment of a display methodoutput that includes a signal time history plot of normalizedfluorescence intensity values on the display;

FIG. 14 illustrates a recorded image of an anatomical fluorescenceimaging phantom, featuring an embodiment of a display method output thatdisplays normalized fluorescence intensity;

FIG. 15 illustrates an exemplary light source of an exemplaryillumination source of the system for illumination shown in FIG. 1;

FIG. 16 illustrates an exemplary imaging module of the fluorescenceimaging system in FIG, 1, the imaging module comprising a camera module;

FIG. 17A illustrates a perspective top view of an illumination andimaging system according to an embodiment;

FIG. 17B illustrates a schematic side view of a movable filter assemblyfor the illumination and imaging system of FIG. 17A, according to anembodiment;

FIGS. 17C to 17D illustrate an enclosure according to an embodiment;

FIG. 17E illustrates a sensor and light source arrangement in a forwardportion of an enclosure according to an embodiment;

FIG. 18 illustrates a schematic diagram of components of an illuminationand imaging system according to an embodiment; and

FIG. 19 illustrates a schematic diagram of a drape detection moduleaccording to an embodiment.

DETAILED DESCRIPTION OF THE INVENTION

Example embodiments will now be described more fully hereinafter withreference to the accompanying drawings; however, they may be embodied indifferent forms and should not be construed as limited to theembodiments set forth herein. Rather, these embodiments are provided sothat this disclosure will be thorough and complete, and will fullyconvey exemplary implementations to those skilled in the art. Variousdevices, systems, methods, processors, kits and imaging agents aredescribed herein. Although at least two variations of the devices,systems, methods, processors, kits and imaging agents are described,other variations may include aspects of the devices, systems, methods,processors, kits and imaging agents described herein combined in anysuitable manner having combinations of all or some of the aspectsdescribed.

Generally, corresponding or similar reference numbers will be used, whenpossible, throughout the drawings to refer to the same or correspondingparts.

Spatially relative terms, such as “beneath”, “below”, “lower”, “above”,“upper”, and the like, may be used herein for ease of description todescribe one element or feature's relationship to another element(s) orfeature(s) as illustrated in the figures. It will be understood that thespatially relative terms are intended to encompass differentorientations of the device in use or operation in addition to theorientation depicted in the figures. For example, if the device in thefigures is turned over, elements described as “below” or “beneath” otherelements or features would then be oriented “above” the other elementsor features. Thus, the exemplary term “below” can encompass both anorientation of above and below. The device may be otherwise oriented(rotated 90 degrees or at other orientations) and the spatially relativedescriptors used herein interpreted accordingly.

FIG. 1 illustrates a schematic view of an illumination and imagingsystem 10 according to an embodiment. As may be seen therein, the system10 may include an illumination module 11, an imaging module 13, and avideo processor/illuminator (VPI) 14. The VPI 14 may include anillumination source 15 to provide illumination to the illuminationmodule 11 and a processor assembly 16 to send control signals and toreceive data about light detected by the imaging module 13 from a target12 illuminated by light output by the illumination module 11. In onevariation, the video processor/illuminator 14 may comprise a separatelyhoused illumination source 15 and the processor assembly 16. In onevariation, the video processor/illuminator 14 may comprise the processorassembly 16 while one or more illumination sources 15 are separatelycontained within the housing of the illumination module 11. Theillumination source 15 may output light at different waveband regions,e.g., white (RGB) light, excitation light to induce fluorescence in thetarget 12, a combination thereof and so forth, depending oncharacteristics to be examined and the material of the target 12. Lightat different wavebands may be output by the illumination source 15simultaneously, sequentially, or both. The illumination and imagingsystem 10 may be used, for example, to facilitate medical (e.g.,surgical) decision making e.g., during a surgical procedure. The target12 may be a topographically complex target, e.g., a biological materialincluding tissue, an anatomical structure, other objects with contoursand shapes resulting in shadowing when illuminated, and so forth. TheVPI 14 may record, process, display, and so forth, the resulting imagesand associated information.

FIG. 2 illustrates a schematic perspective view of the illuminationmodule 11 of FIG. 1 according to an embodiment. As may be seen therein,the illumination module 11 may include at least two illumination portsdirecting illumination from an illumination source 23, which may beincluded in the VPI box 14, to for example a rectangular target field24. In some variations, the illumination source 23 may be located in adevice housing along with the illumination module 11. Each illuminationport is to provide illumination over the target field 24, such that thelight overlaps, e.g., substantially or completely, at the targetmaterial 12 (shown in FIG. 1). More than two illumination ports may beused. The illumination distributions may be substantially similar andoverlap (e.g., substantially or completely) at the target 12 to provideuniform illumination of the target 12. The use of at least twoillumination ports facilitates reducing the effect of shadowing due toanatomical topography, and aids in providing uniform illumination overthe target field 24. Directing illumination from the illumination module11 to a rectangular target field 24 (which may have a configurationother than rectangular in other embodiments) allows matching the regionof illumination to a rectangular imaging field of view (which may have aconfiguration other than rectangular in other embodiments), which aidsin providing uniform illumination and may enhance efficiency of theillumination module by reducing extraneous illumination. Matching theillumination field to the imaging field of view also provides a usefulindication of the location and extent of the anatomical region currentlybeing imaged. In some variations, illumination from the illuminationmodule 11 may be directed to provide uniform illumination of the target12 without matching the region of illumination to a rectangular imagingfield of view, and the rectangular target field 24 of FIG. 2 may bereplaced by a non-rectangular target field.

In some embodiments, a light pipe may be used to achieve mixing of theillumination light in order to yield a uniform illumination profile.Mixing of the illumination light by a light pipe may remove theinfluence of the structure of the light source on the illuminationprofile, which could otherwise adversely affect uniformity of theillumination profile. For example, using a light pipe to mix theillumination light output from a fiber optic light guide may removeimages of the structure of the individual optical fibers from theillumination profile. In some embodiments, a rectangular light pipe maybe used to efficiently utilize illumination power while matching theillumination profile to a rectangular imaging field of view. In someembodiments, a light pipe material with a high index of refraction forboth visible light and near infrared light, such as optical glassmaterial N-SF11, may be used for high efficiency of illumination powertransmission.

According to some embodiments, a rectangular light pipe with an aspectratio matching the aspect ratio of the imaging field of view (e.g., bothaspect ratios being 16:9) may be used in conjunction with rotationallysymmetric illumination optic elements.

According to some embodiments, a rectangular light pipe with a differentaspect ratio than the imaging field of view (e.g., a square light pipealong with a 16:9 imaging field of view aspect ratio) may be used inconjunction with cylindrical illumination optic elements. Cylindricaloptic elements may be used to separately conform one or both dimensionsof the rectangular illumination profile to match the aspect ratio of theimaging field of view.

Depending on the desired system requirements for range of workingdistance and illumination uniformity various approaches may be used formatching the illumination to overlap the imaging field of view. Forexample, applications which require a large range in working distancesand high illumination uniformity may necessitate use of illuminationoptics and/or ports that are steered dynamically to adequately match theillumination to the imaging field of view, while applications with lowerrequirements may be served with fixed illumination optics and/or portsto match the illumination to the field of view.

In some embodiments, the direction of illumination is adjusted frommultiple illumination ports in synchrony with adjustment of the field ofview, in order to steer the field of illumination to maintaincorrespondence to the field of view.

In some embodiments, one or more illumination optic elements may berotated by a driver in order to steer the illumination.

In some embodiments, one or more illumination optic elements may betranslated perpendicular to the imaging optic axis by a driver in orderto steer the illumination.

In some embodiments, one or more illumination optic elements may beconfigured to provide some distortion in the illumination profile, inorder to account for distortion inherent to the accompanying imagingsystem.

In some embodiments, uniform illumination of the imaging field of viewover a specified range of working distances may be achieved with a fixedlocation and orientation of the illumination optics. The offset distanceof the illumination optics from the imaging optic axis may beconfigured, along with the orientation of the of the illuminationoptics, in order to optimize matching of the illumination profile to theimaging field of view at a working distance within the specified rangeof working distances while also maintaining substantial matching of theillumination profile to the imaging field of view at other workingdistances within the specified range.

As is illustrated in FIG. 2, each illumination port may include a lensmodule 20, a connecting cable 22 connected to the illumination lightsource 23, and a light pipe 21 adapting a high numerical aperture of theconnecting cable 22 to a lower numerical aperture of the lens module 20.The lens module 20 may be steerable, as described in detail below. Insome scenarios, acceptable performance may be achievable withoutsteering. In other words, an illumination module, and imaging devicehaving the same, that provides an illumination field having arectangular form factor (or configuration other than rectangular) thatmatches the field of view of the imaging system using at least twoillumination ports in which each port produces a gradient ofillumination such that the sum illumination flux in the object plane isreasonably e same at each point in the illumination field, e.g.,provides uniform illumination over the imaging field of view, alone maybe sufficient.

In some variations in which the illumination light source 23 may becontained within a device housing along with illumination module 11, theconnecting cable 22 from FIG. 2 may be replaced by one or moreillumination light sources 23. In some variations, the connecting cable22 and the light pipes 21 from FIG. 2 may be replaced by one or moreillumination light sources 23. In some variations, the lens module 20from FIG. 2 may contain the illumination light source 23. In somevariations, separate variants of the lens module 20 from FIG. 2 mayseparately contain a white light source and a fluorescence excitationlight source of the illumination light source 23. In one embodiment,three or more lens modules 20 may be arranged to comprise a ring ofillumination ports, another functionally equivalent configuration ofillumination ports, or another configuration including continuous ornon-continuous distribution/arrangement of illumination ports, with eachlens module 20 oriented to converge on and provide uniform illuminationover the imaging field of view. In some variations, the three or morelens modules 20 comprising a ring of illumination ports may notnecessarily constrain illumination to a rectangular field, and therectangular target field 24 of FIG. 2 may be replaced by anon-rectangular target field, such as for example a circular/oval targetfield.

FIGS. 3A and 3B illustrate a side view and a plan view, respectively, ofthe lens module 20. The lens module 20 may include lenses mounted in asteerable lens housing 30. As used herein, a lens is any optical elementhaving optical power, whether implemented by a refractive or diffractiveelement. Other elements not essential to understanding, such as a coverenclosing the lens module (see FIG. 2), are not shown for ease ofillustration.

In the particular example shown herein, the lenses may include a pair ofhorizontal-axis cylindrical lenses 31-32 and a pair of vertical-axiscylindrical lenses 33-34. A prism element 35 is also shown which mayalign illumination light with the intended outgoing optical axis. Inparticular, the prism element 35 corrects for an angle introduced by thelight pipe 21 for increased device compactness in accordance with anembodiment. The mounting design for each lens element 31-35 may allowfor tuning of the magnification and focus of the illumination opticalsystem. In accordance with this embodiment, the steerable lens housing30 encloses and steers three of the cylindrical lenses 31, 33, 34 andthe prism lens element 35, e.g., collectively as a group. This exampleof lenses is merely illustrative, and the lenses in the lens module 20may be modified as appropriate.

In this particular embodiment, a base portion of the steerable housing30 is pinned, e.g., using a pin 46 (see FIG. 6B) inserted into housinghole 37, about a pivot point 36, respectively to a fixed chassis frame90 (see FIG. 6A) and a mechanical linkage 40 (see FIGS. 4A to 4C)described in detail below, while lens 32 is rigidly connected thechassis 90, i.e. not to the housing 30 (see FIG. 6B).

FIG. 4A illustrates a schematic view showing directions of motionprovided by various components of the linkage 40. The linkage 40 mayinclude a drive cam 41, illumination cams 45 a, 45 b (one for eachillumination port), and an imaging cam 43. The drive cam 41 receives aninput from a user (see FIG. 7), and translates that to synchronousmotion of the lens module 20 a, 20 b, attached to a correspondingillumination cam 45 a, 45 b, via a respective housing 30 (see FIG. 3B)and a pin 46 (see FIG. 6B), and an imaging lens 51 and an imaging sensor52. (see FIGS. 5A and 5B), attached to the imaging cam 43 via camfollower pins. Here, the imaging lens 51 is shown as a single fieldlens, but additional and/or alternative lenses for focusing light fromthe target 20 onto the imaging sensor 52 may be employed. Each port hasits own associated illumination cam 45A or 45B, here shown as being to aleft and right of an input window to receive light from the target 12.Here, drive cam 41 is shown as a plate with a front edge extendingbeyond the rear of the lens modules 20 a, 20 b, but the drive cam 41need not be in the form of a plate and may instead comprise multiplesurfaces to interface with and drive three or more lens modules, inwhich case the front edge of the drive cam 41 and the rear edges ofillumination cams 45 a, 45 b may be set further to the rear in order toaccommodate additional lens modules and corresponding illumination cams.

In particular, translation of the drive cam 41 may translate the imagingcam 43 along the x-axis, which, in turn, may result in the imaging cam43 to translate the imaging lens 51 and the imaging sensor 52 along thez-axis, as well as translate the illumination cams 45 a, 45 b, which, inturn, simultaneously steer corresponding lens modules 20 a, 20 b aboutrespective pivot points 36, such that steering of the lens modules 20 a,20 b is synchronously performed with the position adjustment of theimaging lens 51 and the imaging sensor 52 to insure proper focus oflight from the target onto the sensor 52. Alternatively, the imaging cam43 may translate only the imaging lens 51 along the z-axis, or any othercombination of imaging optical elements in order to insure proper focusof light from the target onto the sensor 52.

FIG. 4B illustrates a bottom view and FIG. 4C illustrates a top view ofthe linkage 40 according to an embodiment. The drive cam 41 may includetwo drive parts 41 a and 41 b, and, if steering is included, a thirddrive part 41 c, all of which are shown here as being rigidly attachedto form a rigid drive cam 41. Similarly, the imaging cam 43 may includetwo imaging parts 43 a and 43 b. The drive cam 41 receives the inputfrom a user (via control surface 62) via the first drive part 41 a andtranslates the imaging cam 43 via a cam follower pin in drive part 41 b,resulting in the imaging cam part 43 a translating the sensor 52 and theimaging cam part 43 b translating the imaging lens 51. If steering isincluded in the linkage, the third drive part 41 c simultaneously steers(rotates) the lens modules 20 a, 20 b using the pin 46 (see FIG. 6B)associated with each of the illumination cam parts 45 a and 45 b, bytranslating the illumination cam parts 45 a and 45 b. The pin 46 may beinserted through a through a slot 49 in each of the illumination cams 45a, 45 b and the corresponding housing hole 37 in the lens modules 20 a,20 b. The drive part 41 c steers the lens modules 20 a, 20 bsimultaneously such that they both still illuminate a same field of viewas one another at the target field of view of the target 12.

FIGS. 5A and 5B illustrate bottom views of the linkage combined with thelens modules 20 a, 20 b, the imaging field lens 51, and the sensor 52,at a far working distance and a near working distance, respectively,according to an embodiment. As can be seen therein, the linkage 40synchronizes steering of the illumination sources with focusing of theimaging system at two sample working distance illumination steeringsettings. FIGS. 5A-5B show the positions of lens modules 20 a, 20 b(rotated about the pivot pint 37) and the lens 51 and sensor 52(translated along an optical axis 55 of the imaging system and along thex-axis) at two focus positions resulting from user input.

As illustrated in FIGS. 5A and 5B, each part that moves axially withinthe linkage mechanism 40 may be guided by two fixed rolling elements 47,and one spring-loaded rolling element 48, in order to reduce or minimizefriction during motion. The linkage 40 also may include a drive caminput connection point 42.

FIGS. 6A and 6B illustrate a perspective top view and a perspectivebottom top view of the device 10 in accordance with an embodiment. InFIGS. 6A and 6B, the illumination module 11 and the imaging module 13are mounted on the chassis 90, the top portion of which is removed infor clarity. Also, a focus actuation mechanism 70 is illustrated, whichtranslates motion from user input to motion of the drive cam 41 via thedrive cam input connection point 42.

As can be seen in FIG. 6A, an optical axis 55 of the imaging module 13runs through a center of the imaging module, with the lens modules 20 a,20 b being arranged symmetrically relative to the imaging optical axis55. The light to be imaged from the target 12 travels along the opticalaxis 55 to be incident on the lens 51 and sensor 52. Awavelength-dependent aperture 53 that includes a smaller centralaperture that permits transmission of all visible and fluoresced light,e.g., near infrared (NIR) light, and a surrounding larger aperture thatblocks visible light but permits transmission of fluoresced light, maybe provided upstream of the lens 51.

Referring to FIGS. 6B and 4A-4B, the pin 46 connects the lens module 20,via the housing hole 37 in the housing 30, slot 49 of the linkage 40.Also, a pivot point pin 44 connects the lens module 20 to the chassis90.

FIG. 7 illustrates an embodiment of an ergonomic enclosure 60 enclosingthe illumination module 11 and the imaging module 13. The ergonomicenclosure 60 is designed to be held in differentuse-modes/configurations, for example, a pistol-style grip for forwardimaging in a scanning-imaging orientation (FIG. 8A), and avertical-orientation grip for use when imaging downward in an overheadimaging orientation (FIG. 8B). As may be seen in FIG. 7, the enclosure60 includes a control surface 62, a grip detail 64, a window frame 68and a nosepiece 66. The ergonomic enclosure 60 is connectable to the VPIbox 14 via a light guide cable 67, through which the light is providedto the illumination ports 20 a, 20 b, and a data cable 65 that transmitspower, sensor data, and any other (non-light) connections.

The control surface 62 includes focus buttons 63 a (decreasing theworking distance) and 63 b (increasing the working distance) thatcontrol the linkage 40. Other buttons on the control surface 62 may beprogrammable and may be used for various other functions, e.g.,excitation laser power on/off, display mode selection, white lightimaging white balance, saving a screenshot, and so forth. Alternativelyor additionally to the focus buttons, a proximity sensor may be providedon the enclosure and may be employed to automatically adjust the linkage40.

As can be seen in FIG. 8A, when the enclosure 60 is held with theimaging window facing forward, the thumb rests on the control surface 62while the other fingers on the operator's hand are wrapped looselyaround the bottom of the grip detail 64. As can be seen in FIG. 8B, whenthe enclosure 60 is held with the imaging window facing downward, thegrip detail 64 is between the thumb and index finger and the fingers arewrapped around to access the control buttons or switches on the controlsurface 62. The grip detail 64 is sculpted so as to provide for partialsupport of the device weight on the top of the wrist in thevertical-orientation grip, such that the enclosure 60 can hang looselyand without the need for a tight grip of the enclosure 60. Thus, theenclosure 60 may be operated by a single hand in multiple orientations.In various other embodiments, the enclosure 60 may be supported on asupport (e.g., a movable support).

The window frame 68 (see also FIG. 9A), defines the different windowsfor the enclosure 60. In other words, the window frame 68 defineswindows 68 a and 68 b, corresponding to the two lens modules 20 a and 20b, as well as window 68 c, which serves as an input window for lightfrom the target to be incident on the sensor 52.

FIGS. 17A-D illustrate an imaging system 300 in accordance with oneembodiment. Imaging system 300 may include one or more components ofimaging system 10 of FIG. 1. For example, imaging system 300 maycomprise illumination module 11 and imaging module 13 of system 10.System 300 may be used for or with any of the methods and processesdescribed herein with respect to system 10.

As shown in FIG. 17A, which is a perspective top view, imaging system300 includes two illumination ports 311, imaging module 313, and plate302, each of which is mounted to a frame or chassis (not shown). Thelight to be imaged from target 12, which may include light fromillumination ports 311 reflected by target 12 and/or fluorescent lightemitted from target 12, travels along the optical axis 355, throughplate 302 and into imaging module 313, which houses one or more imagingsensors. As described below, imaging module 313 may include movablefilters for filtering light that enters the imaging module. In someembodiments, the imaging module 313 may include one or morewavelength-dependent apertures that includes a smaller central aperturethat permits transmission of all visible and fluoresced light, e.g., NIRlight, and a surrounding larger aperture that blocks visible light butpermits transmission of fluoresced light.

Each illumination port 311 includes a lens module 320, a connectingcable 322 connected to the illumination light source 23, and a lightpipe 321 adapting a high numerical aperture of the connecting cable 322to a lower numerical aperture of the lens module 320. The lens modules320 may provide illumination having a rectangular form factor thatmatches the field of view of the imaging system 300. Each illuminationport 311 may produce a gradient of illumination such that the sumillumination flux in the object plane is reasonably the same at eachpoint in the illumination field, e.g., providing uniform illuminationover the imaging field of view. Lens modules 320 each include one ormore lenses and/or prism elements for shaping and orienting illuminationto meet application requirements. For example, since the twoillumination ports 311 lie horizontally offset from the center of theoptical axis 355 of the imaging system 300, prisms may be included inthe lens modules 320 to direct the beams towards the center of the fieldof view. The degree of direction may be tailored to the specificapplication, or to a set of specific applications. For example, in somevariations, the degree of direction is selected such that the beamsoverlap at a nominal imaging distance of 25 cm. In some variations, thehorizontal offset of the illumination ports 311 and the degree ofdirection are selected such that the beams substantially overlap andsubstantially cover the field of view over a range of working distances,such as distances from 18-40 cm. In the embodiment illustrated in FIG.17A, the illumination ports are fixed with respect to the frame. Inother embodiments, the illumination ports are steerable, in accordancewith the principles described above.

Imaging module 313 includes image sensor assembly 352, optics module351, and movable filter assembly 330 aligned along an optical axis 355.The image sensor assembly 352, which includes an image sensor and mayinclude one or more lenses, filters, or other optical components, ismovable relative to the frame along the optical axis 355 via focusactuation assembly 370. Focus actuation assembly 370 includes lead nut372 affixed to the housing of the image sensor assembly 352. The leadnut 372 is coupled to lead screw 374, which extends from focus motor376. Focus motor 376 is fixed to the frame and can be actuated inforward and reverse directions to turn lead screw 374, which causes leadnut 372 to translate along the lead screw axis, moving image sensorassembly 352 forward and backward along the optical axis 355. Lead nut372 and/or focus actuation assembly 370 may be mounted on shafts thatslide within mountings on the frame, for example, using one or morelinear ball bearings or bushings to restrain lateral and angular play.In some embodiments, the image sensor assembly 352 may comprise a singleimage sensor that is configured to detect light from the targetresulting from illumination by visible light and excitation light. Inother embodiments, the image sensor assembly 352 may comprise multipleimage sensors for. For example, the image sensor assembly 352 maycomprise separate image sensors configured to detect light from thetarget resulting from illumination by visible light separately from thatresulting from illumination by excitation light.

A controller may be used to control movement of the image sensorassembly 352 for focusing, which may be based on user input. Forexample, system 300 may be provided with one or more controls such asbuttons or touch screen controls to enable the user to adjust the focus.A user may actuate a focus control until the desired focus is achievedor may enter a value associated with a desired focus and the controllermay actuate the image sensor assembly 352 until the desired focus isachieved. In some embodiments, a magnetic position sensor mounted on thehousing of the image sensor assembly 352 detects the position of theimage sensor assembly 352 for closed loop control of focus actuationassembly 370 by the controller. In some embodiments, the controller canuse open loop control of focus actuation assembly 370, for example, byusing a stepper motor.

Optics module 351, which is located forward of image sensor assembly352, is fixed relative to the frame and may include one or more opticalcomponents (e.g., lenses, apertures, filters, etc.) for adjusting lighttraveling along the optical path before reaching the image sensor. Forexample, optics module 351 may include a wavelength-dependent aperture(e.g., similar to aperture 53 of FIG. 6A) that includes a smallercentral aperture that permits transmission of all visible and fluorescedlight, e.g., NIR light, and a surrounding larger aperture that blocksvisible light but permits transmission of fluoresced light.

Movable filter assembly 330 is located forward (upstream with respect tothe direction of travel of light from a target to the image sensor) ofoptics module 351 and includes first window 334 a and second window 334b, each of which is housed in a bracket (first window bracket 332 a andsecond window bracket 332 b, respectively). First and second windows 334a, 334 b can be alternately moved into and out of the optical path. Insome embodiments, the first and second windows 334 a, 334 b can bealternately moved into and out of the optical path via linkage assembly336, which is actuated by filter motor 338. In some variations, thefirst and/or second windows can be moved via any combination of motionsincluding rotation (for example on a rotary wheel) and/or translation.One or both of the windows 334 a, 334 b can include filters forfiltering light before it reaches the image sensor. By moving filtersinto and out of the optical path, imaging system 300 can be operated indifferent imaging modes. For example, in some embodiments, one of thewindows (e.g., first window 334 a) includes a filter for blockingvisible light while the other window (e.g., second window 334 b)includes a clear glass plate that does not block light. With theblocking filter in the optical path, the imaging system can be operatedin a first mode and, with the clear glass in the optical path, theimaging system can be operated in a second mode. When switching modes,one window moves into the optical path while the other window moves outof the optical path. In some embodiments, a visible-light rejectionfilter which only transmits NIR light between 830-900 nm is included ina first window for a fluorescence-only imaging mode and ananti-reflective coated glass plate, which passes all light, is includedin the second window for use in a second mode. The glass plate canensure the same optical path length regardless of mode. In somevariations, a controller of system 300 can control movable filterassembly 330 to change modes, for example, in response to a user input.

In the configuration illustrated in FIG. 17A, second window bracket 332b is in a deployed position such that second window 334 b is positionedin the optical path and is oriented perpendicularly to the optical axis355. By actuating filter motor 338, which actuates linkage assembly 336,the second window bracket 332 b and second window 334 b move out of theoptical path by pivoting about a pivot axis that extends perpendicularlyto optical axis 355. At the same time, first window bracket 332 a andfirst window 334 a move into the optical path by pivoting about a pivotaxis that extends perpendicularly to optical axis 355. In someembodiments, the pivot axis of the first window bracket and the pivotaxis of the second window bracket are vertically aligned and the firstand second window brackets and window are symmetrical to providematching optical path lengths regardless of mode.

Linkage assembly 336 is actuated by filter motor 338, which may becontrolled by a controller of system 300. Filter motor 338 rotatesfilter lead screw 341, which moves filter lead nut 342 forward andrearward. Linkage 344 is pivotally connected on a first end to filterlead nut 342 and pivotally connected at a second end to slider 346 a. Apivot link 348 a is pivotally connected at one end to slider 346 a andat the other end to first window bracket 332 a. As illustrated in FIG.17B, slider 346 b and pivot link 348 b (which are not shown in FIG. 17A)are provided below slider 346 a and pivot link 348 a for actuatingsecond window bracket 332 b.

Movable filter assembly 330 is schematically depicted in FIG. 17B.Filter motor 338, which is fixed relative to the frame, rotates thefilter lead screw 341 clockwise and counterclockwise, causing filterlead nut 342 to translate forward and rearward along the filter leadscrew axis. Translation of filter lead nut 342 causes translation ofslider 346 a via linkage 344. Translation of slider 346 a causestranslation of pivot link 348 a. Pivot link 348 a is pivotally connectedto first window bracket 332 a at a location off-center from the pivotconnection 349 a of first window bracket 332 a to the frame. Therefore,movement of pivot link 348 a causes rotation of first window bracket 332a. For example, from the configuration of FIG. 17B, translation ofslider 346 a forward (toward plate 302) causes first window bracket 332a to rotate 90 degrees out of the optical path.

Driving linkage 345 is pivotally connected at a first end to linkage344, pinned to the frame at connection point 345 a, and pivotallyconnected at a second end to slider 346 b. Thus, translation of linkage344 causes rotation of driving linkage 345, which translates slider 346b. Slider 346 b is connected to second window bracket 332 b via pivotlink 348 b, which is pivotally connected to second window bracket 332 bat a location off-center from the pivot connection 349 b of secondwindow bracket 332 b to the frame. Thus, translation of slider 346 bcauses rotation of second window bracket 332 b. From the configurationof FIG. 17B, translation of slider 346 b rearward (as slider 346 a movesforward), causes second window bracket 332 b to rotate 90 degrees intothe optical path. One or more sensors may be included for sensing theposition of one or more of the movable filter assembly 330 componentsfor providing feedback to the controller for closed-loop control.

Plate 302 is a flat plate for sealing the housing and protecting theillumination and imaging optics. In some embodiments, plate 302 is asingle plate of glass. One or more optical components such as a lens maybe mounted between the glass plate and the movable filter assembly 330.In some variations, one or more sensors are positioned on the rear sideof plate 302 to measure light incident on plate 302. One or more ofthese sensors may detect ambient light, light reflected from the target,light emitted by the target, and/or light reflected from non-targetobjects. In some embodiments, a drape detector is included to detect thepresence of a drape. The drape detector may include, for example, aninfrared emitter and a photodetector that detects infrared lightreflected by a drape positioned on the imaging system.

FIGS. 17C-D illustrate an embodiment of an ergonomic enclosure 360enclosing illumination ports 311 and imaging module 313, according toone variation. The ergonomic enclosure 360 is designed to be held in apistol-style grip. The enclosure 360 may include a control surface 362,a grip 364, a window frame 368 and a nosepiece 366. The ergonomicenclosure 360 is connectable to the VPI box 14 via a light guide cable367, through which the light is provided to illumination ports 311, anda data cable 365 that transmits power, sensor data, and any other(non-light) connections.

The control surface 362 includes focus buttons 363 a and 363 b thatcontrol the focus actuation assembly 370. Other buttons on the controlsurface 362 may be programmable and may be used for various otherfunctions, excitation laser power on/off, display mode selection, whitelight imaging white balance, saving a screenshot, and so forth.Alternatively or additionally to the focus buttons, a proximity sensormay be provided on the enclosure and may be employed to automaticallyadjust the focus actuation assembly 370.

Enclosure 360 may be operated by a single hand in a pistol-grip styleorientation. In various other embodiments, the enclosure 360 may besupported on a support (e.g., a movable support). In some embodiments,enclosure 360 may be used in concert with a drape, such as drape 80 ofFIG. 9A or drape 390 of FIG. 9B.

In some embodiments, a window frame 368 is provided on the forwardportion of enclosure 360 in front of plate 302. In other embodiments,the window frame 368 is provided on the forward portion of enclosure 360behind plate 302, and plate 302 provides the outer surface of theenclosure. In other embodiments, no frame is provided and plate 302provides the outer surface of the enclosure. Window frame 368 mayinclude windows 368 a and 368 b, corresponding to the two lens modules320, as well as window 368 c, which serves as an input window for lightfrom the target to be incident on the image sensor. Window frame 368 mayalso include one or more windows 369 for sensors provided behind plate302.

FIG. 17E illustrates an embodiment of a sensor arrangement providedbehind plate 302 on the forward portion of enclosure 360, according toone variation. In this embodiment, a central sensor group 391 comprisingone or more sensors 392 is provided in order to detect reflectedillumination light for input to an automatic gain control function, asdescribed below. Also in this embodiment, peripheral sensor groups 393 aand 393 b, each comprising one or more sensors 394, are provided inorder to detect reflected illumination light for purposes of proximitydetection to the imaging target or to detect any objects near to theforward portion of the enclosure 360, as described below. The source ofthe illumination light for proximity detection may be either the mainillumination beam or may be one or more dedicated emitters for proximitydetection. Also in this embodiment, one or more sensors 387 and one ormore light sources 386 are provided in order to detect the presence ofan installed drape lens, as described below. Also in this embodiment,one or more sensors 395 may be provided in order to detect ambient roomlight intensity to facilitate correction of image intensity artifactsarising from pulsating room light components, as described herein.

The sensors 392 may be used to detect reflected light levels in order toprovide input for an automatic gain control (AGC) function (see FIG. 18)that may be used to facilitate optimizing illumination and imagingparameters and providing a consistent and/or smoothly varying imagebrightness, even when varying the working distance. AGC may also be usedto facilitate optimizing or maximizing the image signal to noise ratio,or to minimize the illumination intensity to facilitate minimizingphoto-bleaching. For example, the AGC may be used to dynamically adjustimage signal gain, illumination pulse duration, exposure, and/orillumination power. The reflected illumination light detected by thesensors 392 may include visible light and/or fluorescence excitationlight, such as NIR light. In one embodiment, sensors 392 are sensitiveto NIR light but not to visible light, such that ambient visible lightand white light illumination do not contribute to the light level signalfrom sensors 392. In some variations, sensors 392 are comprised ofphotodiodes.

The reflected light level sensors 392 may be used as input to AGC in anyimaging mode, including a white light imaging mode and/or a multiplexedcombined white light and fluorescence imaging mode, and may beparticularly important in a fluorescence-only imaging mode. Whenoperating in a fluorescence-only imaging mode, for example with filter334 a blocking visible light from reaching the image sensor, noreflected white light luminance image is recorded, which could otherwisebe used as an input to AGC, while the recorded fluorescence imagenecessarily excludes reflected fluorescence excitation light (whichwould otherwise overpower the fluorescence signal) through use of anotch filter in the imaging optics. Therefore, the sensors 392 mayprovide the only measure of reflected light. In one variation, theoperation of AGC in a fluorescence-only imaging mode prioritizesmaximizing the exposure duration and minimizing the gain.

In some embodiments, for which the sensors 392 are sensitive to theexcitation light, the gain, excitation period (which may, for example,be the same as the image sensor exposure time) and instantaneousexcitation power can be adjusted as follows in order to achieve aconstant image brightness for a given fluorescence sample regardless ofworking distance. Based on the reflected excitation light E, as measuredby sensors 392, the AGC may adjust the excitation period, T,instantaneous excitation power, P, and image sensor gain, G, such that F*T*G=K, where K is a constant based on the desired target brightness.The priorities of adjusting T, G and P can be optimized to minimizenoise while limiting maximum exposure of tissue to excitation light.

In one embodiment, as shown in FIG. 17E, the sensors 392 are arrangedsuch that their detection cones approximately cover the imaging field ofview. For example, in this embodiment, a sensor group 391 is comprisedof four sensors 392 arranged in a rectangular pattern surrounding theimaging port.

According to one embodiment, AGC operates by starting with settings foran initial gain g₀, initial exposure e₀, and initial illumination powerp₀. User defined brightness parameters may prescribe target values, suchas for example, a target peak brightness P_(t) and a target meanbrightness M_(t), as well as a choice of AGC mode to be based on thepeak values, mean values, or a balanced combination of both peak andmean values.

During each image acquisition frame, a peak sensor brightness P_(s) maybe calculated based on a peak signal from among sensors 392 during theacquisition duration, and a mean sensor brightness M_(s) may becalculated based on a mean of the signals from sensors 392 during theduration, An adjustment factor F is then calculated based on thesevalues and used to calculate a target exposure value e_(t) and a targetgain value g_(t). For example, in peak mode F=P_(t)/P_(s), in mean modeF=M_(t)/M_(s), and in balanced mode F=(½)(P_(t)/P_(s)+M_(t)/M_(s)). Inone variation, a balanced mode may be a weighted combination of sensorsignal values, such as a weighted average of P_(s) and M_(s) as inF=(k1*P_(s)+k2*M_(s)), where k1 and k2 are constants. In one variation,the constants k1 and k2 may satisfy the constraints k1+k2=1 and0<=k1<=1. The target exposure is calculated as e_(t)=Fe₀, and the targetgain is calculated as g_(t)=Fg₀.

According to an embodiment, AGC adjusts the exposure duration (and thecorresponding excitation illumination duration) by a step equal toone-half of the value between the current exposure e₀ and the targetexposure e_(t), such that the new exposure e₁=e₀+(e_(t)−e₀)/2. In thismanner, the exposure cannot be increased above a maximum exposuree_(max) or decreased below a minimum exposure e_(min).

According to an embodiment, if the current exposure e₀ is at the maximumexposure e_(max) and the adjustment factor F is greater than unity, thenAGC adjusts the gain to a new gain g₁=g₀+(g_(t)−g₀)/4. If the currentgain is greater than unity and F is less than unity, then the gain isinstead adjusted to a new gain g₁=g₀−(g₀−g₀(e_(max)/e₀))/4. Otherwise,the new gain instead remains unchanged as g₁=g₀.

According to an embodiment, the excitation power may be adjusted as alowest adjustment priority.

Following each AGC cycle, the new values for exposure, gain, and powerare treated as the current values for the next AGC cycle.

The sensors 394 may be used to detect reflected illumination light thatis reflected off of objects entering into the periphery of theillumination beams and located near to the front of the enclosure 360.For example, detection of such near objects may be used to triggerswitching to a reduced illumination power setting in order to reduce apossible safety risk from high illumination power being delivered to anearby object. The reflected illumination light detected by sensors 394may include visible light and/or fluorescence excitation light, such asNIR light. In one embodiment, sensors 394 are sensitive to NIR light butnot to visible light, such that ambient visible light and white lightillumination do not contribute to the detection signal from sensors 394.In some variations, sensors 394 are comprised of photodiodes or oftime-of-flight sensors. In one variation, the sensors 394 are arrangedsuch that they may detect objects entering the illumination beams whichare not within the imaging field of view.

In some embodiments, a method for imaging a target includes illuminatingthe target with an illuminator of an imaging system, such asillumination ports 311 of imaging system 300, and receiving light fromthe target at an imaging sensor of the imaging system in an unrestrictedimaging mode. In some embodiments the light received from the targetinclude light reflected by the target and light emitted by the target.In some embodiments, the reflected light includes visible light and theemitted light includes fluorescent light from the target. The imagingmode is switched from the unrestricted imaging mode to a restrictedimaging mode in which light of wavelengths outside of a desiredwavelength band or bands is blocked from reaching the imaging sensor.The light is blocked using a movable filter of the imaging device. Thelight that is passed by the filter is received by the imaging sensor.The imaging mode can be switched back to the unrestricted imaging modein which the filter is moved out of the optical path so that it nolonger blocks light in the optical path.

For example, system 300 can be operated in an unrestricted imaging modein which first window 334 a is in a deployed position in the opticalpath. First window 334 a may include a clear plate that permits alllight to pass through it. In this unrestricted imaging mode, the imagesensor may receive all or most of the light that reaches first window334 a. System 300 can be switched to a restricted imaging mode in whichthe first window 334 a is in a stowed position out of the optical pathand the second window 334 b is in a deployed position in the opticalpath, according to the principles described above. The second window 334b may include a filter that filters out light that is not in a desiredwavelength band or set of wavelength bands. For example, the filter mayfilter out all visible light but pass infrared light (e.g., NIR light).Thus, during the restricted imaging mode, the imaging sensor providesimaging data of only the light passed by the filter.

System 300 may be switched to the restricted imaging mode in response toa request that may be received from a user (e.g., via actuation of oneor more buttons on the control surface 362) or that may be received froman external control system. Although the above description refers torestricted and unrestricted modes, the same principles can be used toswitch between two restricted modes (i.e., sonic light is blocked inboth modes). For example, the system can switch between two restrictedimaging modes by including a first filter configured to block a firstwavelength band or set of wavelength bands and a second filter,different from the first, that is configured to block a differentwavelength band or set of wavelength bands from the first.

In some embodiments, the automatic gain control process described abovecan be started upon switching to the restricted imaging mode and may bestopped upon switching to the unrestricted imaging mode (e.g., AGC canbe automatically started and stopped by a controller of the system 300).In other embodiments, AGC is performed during both the restricted andunrestricted imaging modes.

As illustrated in FIG. 9A, the enclosure 60 of FIG. 7 may be used inconcert with a drape 80. The drape 80 may be a surgical drape suitablefor use during a surgical procedure. The drape includes drape material81, a drape lens 82, a drape window frame 83 surrounding the drape lens,and an interlock interface 84 that is integral with the drape windowframe 83. The drape material 81 is to envelope the device in theenclosure 60, as well as to cover anything else as required. The drapewindow frame 83 may follow a shape of the enclosure nosepiece 66 suchthat the drape window frame 83 may be inserted therein withoutobstructing the windows 68 a to 68 c. The drape 80 is designed tominimize reflections and imaging ghosting by ensuring the drape lens 82is flush, e.g., to within 0.5 mm, with the imaging and illuminationwindow frame 68. The drape 80 may use the interlock interface 84, whichmay fit over a ridge on the inner surface of the enclosure nosepiece 66,to be secured flush thereto. In one variation, the interlock interface84 may fit into a recess on the inner surface of the enclosure nosepiece66.

One or more interlock interfaces 84 may be used on the inner or outersurface of the enclosure nosepiece 66, in order to ensure a secure andclose fit of the drape lens 82 against the window frame 68. In theparticular embodiment shown, two interfaces 84, here one on the top andone on the bottom of the drape window frame 83 to engage with an innersurface of the enclosure nosepiece 66, are used.

According to some variations, feedback may be provided to the user toindicate when the drape lens has been installed correctly onto theenclosure nosepiece. In one variation, a raised ridge around at least aportion of the drape window frame may provide tactile and/or auralfeedback when pushed over one or more detent features on the interiorsurface of the enclosure nosepiece. In another variation, a raised ridgearound at least a portion of the drape window frame may provide tactileand/or aural feedback when pushed over one or more detent features onthe exterior surface of the enclosure nosepiece. In another variation,one or more interlock interfaces may provide tactile and/or auralfeedback when pushed into place to engage with an inner surface of theenclosure nosepiece. In another variation, one or more interlockinterfaces may provide tactile and/or aural feedback when pushed intoplace to engage with an outer surface of the enclosure nosepiece.Additionally or alternatively, a drape detection module, as describedbelow, may provide feedback to indicate when the drape lens has beeninstalled correctly.

According to an embodiment, the drape may be symmetrical such that itmay be rotated by 180 degrees about its central axis (e.g., the axisaligned with the imaging optical axis) and may be installed correctlyonto the enclosure nosepiece both before and after such a rotation.

The drape lens material may comprise a transparent polymer material suchas, for example, polymethyl methacrylate (PMMA), polycarbonate,polyvinyl chloride, or polyethylene terephthalate glycol-modified. Inone embodiment, the drape lens material may be chosen based in part onhaving a relatively low refractive index and high light transmission inthe visible and NIR bands compared to other candidate materials, so asto minimize artifacts caused by reflections at the drape lens and tomaximise illumination and imaging transmission. For example, the drapelens material, such as PMMA, may have an index of refraction of lessthan about 1.5 and light transmission greater than about 92% in thevisible and NIR bands. The drape lens and/or the drape window frame maybe manufactured by injection molding.

In one variation, the drape lens may be coated with an anti-reflectioncoating to reduce imaging and illumination artifacts from reflection atthe window.

FIGS. 9B-G illustrate an embodiment of a drape 390 comprising drape lens380 and drape window frame 381 to be used in combination with drapematerial (not shown), such as drape material 81 (see FIG. 9A), to coverthe enclosure 360 of FIG. 17C-D. The drape 390 may be a surgical drapesuitable for use during a surgical procedure. The drape includes drapematerial (not shown), a drape lens 380, a drape window frame 381surrounding the drape lens, an interlock interface 384 that is integralwith the drape window frame 381, and a reflective feature 388. The drapematerial is to envelope the device in the enclosure 360, as well as tocover anything else as required. The drape window frame 381 may follow ashape of a forward portion of the enclosure 360 such that the drapewindow frame 381 may be inserted thereon. The drape 390 is designed tominimize reflections and imaging ghosting by ensuring the drape lens 380is flush, e.g., to within 0.5 mm, with the front surface of theenclosure 360, such as plate 302 of FIGS. 17A-B. The drape 390 may usethe interlock interface 384, which may fit into over a ridge 383 on theinner surface of the front portion of the enclosure 360, to be securedflush thereto.

In one embodiment, a drape detection module may be provided to detectthe installation of the drape lens onto the enclosure nosepiece. Forexample, the drape detection module may use any combination of one ormore ultrasonic sensor, inductive sensor, capacitive sensor, opticalsensor, light emitter, radio frequency identification chip and antenna,hall effect sensor, proximity sensor, or electrical contacts in order todetect the installation of the drape lens onto the enclosure. In oneembodiment, a drape detection light source 386 (see FIG. 17E), such asan LED, may be used to transmit light that is detected by acorresponding sensor 387, such as a photodiode, only when reflected offof an installed drape lens. For example, according to an embodiment, thelight source 386 may have a narrow emission wavelength band centeredaround about 905 nm and the sensor 387 may have a narrow wavelengthdetection band that includes wavelengths of about 905 nm. In oneembodiment, the light source 386 and the sensor 387 are located near theforward portion of the enclosure 360 behind the plate 302. In oneembodiment, as shown in FIG. 9C, a reflective feature 388 is located onthe drape lens 380 in a position aligned with the light source 386 andthe sensor 387, such that light from the light source 386 is reflectedoff of one or more interfaces of the reflective feature 388 and onto thesensor 387. For example, according to one embodiment of a drapedetection module 385 as shown in FIG. 19, the reflective feature 388 maycomprise a triangular prism protruding from the surface of the drapelens 380, which may reflect detection light 389 from light source 386onto sensor 387. For example, the reflective feature 388 may reflectdetection light 389 using total internal reflection. In one variation,the output from the sensor 387 may be fed to a transimpedance amplifierin order to amplify the drape detection signal. In one variation, thelight source 386 and the sensor 387 may be located on the enclosurenosepiece. In one variation, the intensity of the reflected light signaldetected at the sensor 387 may be used as feedback to assess and adjustthe installation positioning of the drape lens 380, in order to minimizeartifacts caused by misalignment of the drape lens. In one variation,detection of the installation of the drape lens 380 as indicated by thedrape detection module 385 may trigger automatic adjustment ofillumination and/or imaging parameters or automated changes to the imageprocessing performed by the processor assembly. For example, the imagingand illumination system may be calibrated and/or configured to correctfor distortion, attenuation, or other effects on illumination and/orimaging caused by the installation of the drape lens 380.

According to an embodiment, the process for installation of the drapeonto the enclosure includes unpacking the drape, installing the drapelens onto the enclosure nosepiece by pushing the drape lens into placeuntil an audible and/or tactile click is sensed by the user (indicatingthe interlock interfaces have engaged with the corresponding ridges inthe enclosure nosepiece), rolling the drape bag back over the camera,and securing as needed the drape bag at the front and rear of theenclosure and along the enclosure cables. To remove the drape from theenclosure, the clips on the drape interlock interfaces may be pressedinwards in order to disengage from the ridges and then pulled away fromthe enclosure. In accordance with the above processes, both theinstallation and removal of the drape lens may be performed with onehand in contact with the drape lens.

FIGS. 10A to 10C illustrate typical illumination distributions (fill)relative to a rectangular imaging field of view (outline) for anillumination ring (FIG. 10A), a pair of fixed anamorphic projectionillumination sources (FIG. 10B), a pair of steered anamorphic projectionillumination sources in accordance with an embodiment (FIG. 10C), and asteered illumination ring (FIG. 10D) at working distances of 10 cm (leftcolumn), 15 cm (center column), and 20 cm (right column). FIG. 10Aillustrates use of a ring of illumination ports to minimize shadowing,but does not match illumination to the imaging field of view and may notprovide even illumination at all working distances (e.g. varieddistributions in accordance with distance). FIG. 10D illustrates use ofa steered ring of three or more illumination ports to facilitateminimizing shadowing and providing even illumination when changingworking distance in accordance with an embodiment, but does notconstrain illumination to the imaging field of view. FIG. 10Billustrates anamorphic projection from two illumination sources using,e.g., an illumination lens arrangement featuring cylindrical lenses oran engineered diffuser) that are fixed, thus they are well calibratedfor even illumination that matches the imaging field of view at a fixedworking distance, e.g., 15 cm, but not as even or well matched at otherdistances, whether smaller or greater. As noted above, such illuminationis often acceptable on its own. FIG. 10C illustrates the ability tobetter maintain even illumination and constrain illumination to thefield of view by steering illumination when changing the workingdistance and imaging focus in accordance with an embodiment.

As noted above, the illumination used may include both white light andfluorescence excitation illumination, e.g., from a laser, to excite NIRlight from the target. However, ambient light may interfere with thelight from the target.

FIG. 11A illustrates a timing diagram for white light (RGB) andfluorescence excitation (Laser) illumination, and visible (VIS) and NIRfluorescence (FL) imaging sensor exposures configured to allow ambientroom light subtraction from the fluorescence signal with a singlesensor. As used herein, a white pulse will indicate that the white light(RGB) is illuminating the target and an excitation pulse will indicatethat the laser is illuminating the target.

Exposures of even (Exp 1) and odd (Exp 2) sensor pixel rows are showninterleaved with differing exposure times to facilitate isolation of anestimate of the ambient room light signal component. Such an interleavedexposure read-out mode is offered on some imaging sensors, such as the‘High Dynamic Range Interleaved Read-out’ mode offered on the CMOSISCMV2000 sensor.

Pulsing the white light illumination at 80 Hz brings the frequency ofthe flashing light above that which is perceptible by the human eye orwhich may trigger epileptic seizures. The visible light image exposuremay be longer than, e.g., twice, the RGB illumination to ensure overlapbetween the 60 Hz exposure frame rate and the 80 Hz RGB illuminationpulse. Extra ambient light captured during the visible exposure may beignored, due to the much greater intensity of the RGB illumination pulseand signal from the target 12.

By setting the NIR fluorescence image exposure times Exp 1 and Exp 2 toacquire for one-half frame and one quarter frame periods, respectively,while running the excitation laser only in the last one quarter of everythird frame, the even rows (Exp 1) record one-half frame of ambient roomlight in addition to one quarter frame of NIR fluorescence, while theodd rows (Exp 2) record one quarter frame of ambient room light plus onequarter frame of NIR fluorescence. Performing these fractional exposureswithin each visible or NIR fluorescence frame minimizes motion artifactswhich would otherwise be caused by inserting additional exposure framesinto the frame sequence for the purpose of ambient room lightsubtraction.

With such an acquisition design, an estimate of the ambient room lightcontribution to the image signals can be isolated by subtracting the Exp2 sensor rows of the NIR fluorescence image from the Exp 1 sensor rows(interpolated to match Exp 2 pixel positions), yielding an estimate ofone quarter frame of ambient room light signal. The estimate of onequarter frame of ambient room light signal can then be subtracted fromthe Exp 2 sensor rows of the NIR fluorescence image to yield an estimateof the NIR fluorescence signal with the one quarter frame of ambientroom light removed. The control of the illumination and the exposure maybe performed by the VPI box 14.

In one embodiment, the above room light subtraction method may bealtered in order to accommodate use of a Bayer-pattern color sensor.FIG. 12A illustrates a Bayer pattern arrangement of colored sensorpixels, wherein the even sensor rows and odd sensor rows have differentfilter arrangements (e.g., no red pixels in the even sensor rows and noblue pixels in the odd sensor rows), so the ambient light recorded oneven rows will not be a good estimate of what reached the odd rows overthe same period. However, every row does include green pixel signals,which are also sensitive to NIR fluorescence. Using only the greenpixels, and performing a two-dimensional interpolation from the greenpixel signals to the other pixel locations can yield an estimate of theambient light signal component, and thus also of the NIR fluorescence orvisible light components for the NIR and visible light images,respectively.

In order to calculate the NIR signal value at a given location,calculate the Exp 1 (even row) and Exp 2 (odd row) green pixel valuesnear that location, with one or both of those values needing to beinterpolated. FIG. 12B demonstrates an example wherein at a red pixellocation, the best estimate of the Exp 1 (even row) green value is theaverage of the immediately neighboring green values above and below,while the best estimate of the Exp 2 (odd row) green value is theaverage of the immediately neighboring green values to the left andright.

The following mathematical example serves to illustrate an embodiment ofthe ambient room light subtraction method. If A=ambient light incidentin one quarter frame period, and F=fluorescence incident in one quarterframe period, then:

Exp 1=2A+F

Exp 2=A+F

Solving for yields:

F=*Exp 2−Exp1

In the particular example illustrated in FIG. 11A, a period for thesensing is three frames, the white light pulse and the excitation pulsehave a same duration or width, but different frequencies, the visiblelight is sensed during two frames, e.g., the first two frames, and thefluorescence is sensed for during one frame, e.g., the third or finalframe, for two different exposure times. As shown therein, the visibleexposure time may be twice the duration of the white light pulse, afirst fluorescent exposure times may be equal to the duration of theexcitation pulse, and a second fluorescent exposure time may be pulselonger, e.g., twice, than the excitation pulse. Further, the visibleexposure may have a different frequency than the white light pulse,e.g., visible exposure does not occur with every white light pulse,while the fluorescent exposure may have a same frequency as theexcitation pulse.

Alternative timing and exposure diagrams are discussed below, in which asensor having rows that are all active for a common exposure durationmay be used while still compensating for ambient light using a singlesensor. For example, background light may be directly detected by thesensor when the target is not illuminated. Other variations on pulsing,exposing, and sensing may be apparent to those of skill in the art.

FIG. 11B illustrates an alternative timing diagram for white light (RGB)and fluorescence excitation (Laser) illumination, and visible (VIS) andNIR fluorescence (FL) imaging sensor exposures configured to allowambient room light subtraction from the fluorescence signal with asingle sensor. Exposures for visible light and for fluorescence areshown in sequence along with an exposure to capture the background (BG)image signal due to ambient light. The white light illumination may bepulsed at 80 Hz as described above. The fluorescence excitationillumination may be pulsed at 20 Hz and the pulse duration or width maybe increased, e.g., up to double the white light pulse duration, toenable a longer corresponding fluorescence exposure. If using an imagingsensor with a global shutter, each sensor exposure must terminate withthe read-out period at the end of an imaging frame. An exposure tocapture the ambient light background image signal may be performed atthe end portion of a frame in the absence of any pulsed white light orexcitation light. In the case of acquiring video at a frame rate of 60Hz, as shown in the example in FIG. 11B, a white light illuminationpulse width of one quarter frame duration may be used, along with a onequarter frame duration visible light exposure occurring in frames whenthe end of a white light illumination pulse is aligned with the end ofthe frame.

A scaled image signal recorded during one or more background exposurescan be subtracted from each fluorescence exposure image to remove thecontribution of ambient light from the fluorescence image. For example,the image signal from a one quarter frame duration background exposuremay be scaled up by two times and subtracted from a subsequent imagesignal from a one-half frame duration fluorescence exposure. As anotherexample, a one quarter frame duration background exposure image signalprior to a one-half frame duration fluorescence exposure image signal,and a second one quarter frame background image signal subsequent to thefluorescence exposure, may both be subtracted from the fluorescenceimage signal. Scaling of the image signals from a first and a secondbackground exposure can include interpolation of pixel values from thefirst exposure time point and the second exposure time point to estimatepixel values corresponding to an intermediate time point.

Use of an imaging sensor with high speed read-out that enables highervideo frame acquisition rates may allow for additional exposure periodsto be allocated within an illumination and exposure timing scheme for agiven white light pulse frequency. For example, maintaining an 80 Hzwhite light illumination pulse as above and using a sensor with a highervideo frame acquisition rate such as 120 Hz may allow additional whitelight, ambient background, or fluorescence exposures within a given timeperiod, compared to when using a slower video frame acquisition ratesuch as 60 Hz.

In the particular example illustrated in FIG. 11B, a period for thesensing is three frames, the excitation pulse has twice the width of thewhite light pulse, the visible light is sensed during one frame, e.g.,the first frame, the background light is sensed during one frame, e.g.,the second frame, and the fluorescence is sensed during one frame, e.g.,the third or final frame. Here, a visible exposure time may be equal tothe duration of the white light pulse, the background exposure time maybe equal to the duration of the white light pulse, and the fluorescenceexposure time may be equal to the duration of the excitation pulse.Further, the visible exposure may have a different frequency than thewhite light pulse, e.g., visible exposure does not occur with everywhite light pulse, while the fluorescent exposure may have a samefrequency as the excitation pulse. Finally, the background exposure mayoccur only once within the period.

FIG. 11C illustrates an alternative timing diagram for white light (RGB)and fluorescence excitation (Laser) illumination, and visible (VIS) andNIR fluorescence (FL) imaging sensor exposures configured to allowambient room light subtraction from the fluorescence signal with asingle sensor with a 120 Hz video frame acquisition rate. A white lightpulse frequency of 80 Hz is used, and a white light illumination pulsewidth of one-half frame duration may be used, along with a one-halfframe duration visible light exposure occurring in frames when the endof a white light illumination puke is aligned with the end of the frame.The fluorescence excitation illumination is shown pulsed at 40 Hz with apulse duration of one frame, to enable a higher frequency ofcorresponding fluorescence exposures. An exposure to capture the ambientlight background image signal may be performed at the end portion of aframe in the absence of any pulsed white light or excitation light, suchas an exposure of one-half frame duration occurring in the frame betweena fluorescence exposure and a successive white light exposure as shownin this example embodiment.

In the particular example illustrated in FIG. 11C, a period for thesensing is three frames, the excitation pulse has twice the width of thewhite light pulse, the visible light is sensed during one frame, e.g.,the second frame, the background light is sensed during one frame, e.g.,the first frame, and the fluorescence is sensed during one frame, e.g.,the third or final frame. Here, a visible exposure time may be equal tothe duration of the white light pulse, the background exposure time maybe equal to the duration of the white light pulse, and the fluorescenceexposure time may be equal to the duration of the excitation pulse.Further, the visible exposure may have a different frequency than thewhite light pulse, e.g., visible exposure does not occur with everywhite light pulse, while the fluorescent exposure may have a samefrequency as the excitation pulse. Finally, the background exposure mayoccur only once within the period.

Depending on the intensity of the fluorescence excitation light used,there may be safety considerations limiting the duration and frequencyof excitation light pulses. One approach to reduce the excitation lightintensity applied is to reduce the duration of the excitation lightpulses and the corresponding fluorescence exposures. Additionally oralternatively, the frequency of excitation light pulses (andcorresponding fluorescence exposures) may be reduced, and the read-outperiods which could otherwise be used for fluorescence exposures mayinstead be used for background exposures to improve measurement of theambient light.

FIG. 11D illustrates an alternative timing diagram for white light (RGB)and fluorescence excitation (Laser) illumination, and visible (VIS) andNIR fluorescence (FL) imaging sensor exposures configured to allowambient room light subtraction from the fluorescence signal with asingle sensor with a 120 Hz video frame acquisition rate. A white lightpulse frequency of 80 Hz is used, and a white light illumination pulsewidth of one-half frame duration may be used, along with a one-halfframe duration visible light exposure occurring in frames when the endof a white light illumination pulse is aligned with the end of theframe. The fluorescence excitation illumination is shown pulsed at 20 Hzwith a pulse duration of one frame. An exposure to capture the ambientlight background image signal may be performed at the end portion of aframe in the absence of any pulsed white light or excitation light, suchas a background exposure of one-half frame duration occurring in theframe between a fluorescence exposure and a successive first white lightexposure, and a first background exposure of one frame duration and asecond background exposure of one-half frame duration both occurring inthe frames between the first white light exposure and a successivesecond white light exposure, as shown in this example embodiment.

In the particular example illustrated in FIG. 11D, a period for thesensing is six frames, the excitation pulse has twice the width of thewhite light pulse, the visible light is sensed during two frames, e.g.,the second and fifth frames, the background light is sensed during threeframes, the first, third, and fourth frames, and the fluorescence issensed for during one frame, e.g., the sixth or final frame. Here, avisible exposure time may be equal to the duration of the white lightpulse, the background exposure time may be equal to or twice theduration of the white light pulse, and the fluorescence exposure timemay be equal to the duration of the excitation pulse. Further, thevisible exposure may have a different frequency than the white lightpulse, e.g., visible exposure does not occur with every white lightpulse, e.g., only twice within the period, while the fluorescenceexposure may have a same frequency as the excitation pulse. Finally, thebackground exposure may occur three times within the period for a totalduration equal to four times the duration of the white light pulse.

In some use environments for an open field imaging device, such as thedevice according to the various embodiments described herein, theambient room lighting may comprise light that is pulsating, or periodic,rather than continuous. Such pulsating light components may, forexample, be due to the interaction between some room light sources andan AC frequency of their power source. For example, incandescent lights,some LED lights, some fluorescent lights including fluorescent lightswith low frequency ballasts, or arc lamps may emit pulsating light whenconnected to common 50 Hz or 60 Hz AC mains power or other AC powersources. The presence of pulsating light components in the backgroundlight signal may introduce distracting image intensity artifacts duringacquisition of sequential images, due to sequential exposures receivingdifferent accumulated light intensity contributions from the pulsatinglight components in the background light, therefore it may be useful tocorrect acquired images to reduce or remove such artifacts. Suchcorrection may be useful both with or without also using a room lightsubtraction technique, and may include one or more exemplary techniquessuch as: detecting the AC frequency of the power source for thepulsating light components; modifying the image acquisition frame rate;modifying the exposure durations for fluorescence and/or backgroundlight exposures; measuring the pulsating light intensity during a periodin which the device illumination is turned off; synthesizing a completeperiodic cycle of the pulsating light intensity; identifying the portionof the periodic cycle of the pulsating light intensity coinciding withthe fluorescence and/or background light exposures; calculating afluorescence accumulated ambient light value, FL_(acc), corresponding tothe accumulated ambient light intensity during a fluorescence exposure;calculating a background accumulated ambient light value, BG_(acc),corresponding to the accumulated ambient light intensity during abackground exposure; and scaling the image intensity of a fluorescenceimage or a background image based on a ratio of the respectiveaccumulated light values, FL_(acc) and BG_(acc), and, subtracting thebackground image from the fluorescence image to output a resultantimage.

In some embodiments, the AC frequency, F_(AC), of the power source for apulsating light component of the ambient room lighting may be retrievedfrom the device memory, for example due to a user setting a knownfrequency value during device calibration in a use environment, or maybe detected based on measurements by the imaging device. For example,one or more sensors 395 (see FIG. 17E) may be used to measure theambient light intensity during one or more periods when the device whitelight illumination is turned off and the fluorescence excitationillumination is turned off. In one embodiment, the one or more sensors395 may be photodiodes and may have similar responsivity to the sensorused for fluorescence imaging, such as responsivity to visible and NIRlight, with input cones approximating the field of view of the imagingdevice. As another example, in one variation in which an image sensorresponsive only to NIR light, or an image sensor with separate filtersprovided forward of the image sensor that block visible or other non-NIRlight from reaching the sensor, is used for fluorescence imaging, theone or more sensors 395 may be photodiodes with responsivity only to NIRlight.

The periods of measurement by sensors 395 should be of sufficientduration and number such that they capture, in combination of successivemeasurement periods captured over, at maximum, about the time betweensuccessive fluorescence exposures, portions of the pulsating ambientlight intensity constituting a complete periodic cycle, and such thatthere is at least partial overlap of cycle coverage for successivemeasurement periods in order to assist with synthesizing the periodiccycle of the pulsating ambient light, which may constrain the lowerlimit of frequencies F_(AC) which may be supported. However, frequencyvalues for F_(AC) that are below 30 Hz may not be practical for use withroom lighting as they may induce noticeable and distracting visiblelight flicker in general use. The frequency of the pulsating lightintensity is typically twice that of the corresponding value of F_(AC),since room light sources typically have equivalent response for each ofthe positive and negative voltage halves of an AC cycle.

FIG. 11E illustrates an exemplary timing diagram for white light (RGB)and fluorescence excitation (Laser) illumination, periods of ambientlight measurement by sensors 395, and visible (VIS) and fluorescence(FL) imaging sensor exposures configured to allow ambient room lightsubtraction from the fluorescence signal and correction for pulsatileambient light intensity with a single sensor, according to anembodiment. In this embodiment, the frequency of fluorescence excitationillumination and corresponding fluorescence exposures is 20 Hz, thefrequency of white light illumination is 80 Hz, and ambient lightmeasurement periods are all those periods in which both the white lightillumination and fluorescence excitation illumination are turned off.The timing scheme shown may allow for pulsating ambient light intensitysignals corresponding to all practical frequency values for F_(AC) of 30Hz or greater to be detected based on measurements within the timebetween successive fluorescence exposures, by capturing, in combinationof multiple measurement periods, portions of the pulsating ambient lightintensity constituting a complete periodic cycle, with at least partialoverlap of cycle coverage for the multiple measurement periods. While asimplified pulsatile ambient light intensity profile with a frequency of120 Hz, corresponding to a F_(AC) of 60 Hz, is shown here for reference,the pulsatile ambient light correction technique as described herein maybe used for any arbitrary pulsatile, or periodic, ambient lightintensity profile. As seen here, the sample pulsatile ambient lightintensity profile would yield different contributions of accumulatedambient light intensity for a fluorescence exposure and a backgroundexposure, wherein those differences are not accounted for simply by adifference in exposure duration, because of those exposures capturingdifferent portions of the pulsatile ambient light intensity profile.Other pulsatile ambient light intensity profiles, such as those with afrequency that is not a multiple or a factor of the fluorescenceexposure frequency, may generally also yield different contributions ofaccumulated ambient light intensity from one fluorescence exposure tothe next.

In some embodiments, a minimum sampling rate within each measurementperiod for sensors 395 may be set to at least four times the quotient ofthe anticipated maximum frequency F_(AC) and the measurement period dutycycle in order to allow accurate synthesis of a complete pulsatingambient light intensity cycle with periodic frequency twice that ofF_(AC). In some variations, a higher sensor sampling rate may be used toprovide more measurement points in partial overlap regions and/or tosupport higher possible F_(AC) values. For example, as shown in FIG.11E, with a measurement period duty cycle of 50%, a sensor sampling rateof at least 480 Hz may be used within the ambient light intensitymeasurement periods to support frequency values for F_(AC) of up to 60Hz and corresponding pulsatile ambient light intensity frequencies of upto 120 Hz. Partial overlap of cycle coverage allows comparison ofmeasurements taken from multiple measurement periods in order to detectthe frequency F_(AC) (or the corresponding frequency of the pulsatileambient light intensity), for example by calculating the frequencyF_(AC) (or the corresponding frequency of the pulsatile ambient lightintensity) corresponding with the best temporal alignment, such as byminimizing a measure of average error between corresponding measurementpoints in candidate temporal alignments, of the portions of the periodiccycle captured by the multiple measurement periods. Arranging theportions of the periodic cycle captured by the multiple measurementperiods according to the best temporal alignment may then yield thesynthesis of a complete periodic cycle of duration 1/(2F_(AC)). In somevariations, a complete periodic cycle may be extracted directly from asingle measurement period which is longer in duration than the completeperiodic cycle. Synthesis or extraction of the complete periodic cyclepermits extending/extrapolating the pulsatile ambient light signalbeyond periods in which measurement by the sensors 395 was performed.

In some embodiments, the imaging device image acquisition frame rate maybe set to match the known or detected AC frequency, or a multiplethereof, of the power source for a pulsating light component of theambient room lighting such that equivalent contributions from thepulsating light component are present in each fluorescence exposure of agiven duration. To accommodate such a setting of the image acquisitionframe rate, corresponding scaling may be performed of the frequency of apulsed white light source, the frequency of a pulsed fluorescenceexcitation light source, and the frequencies of image exposures. Inembodiments also using a room light subtraction technique that includestaking a background light exposure, exposure durations for thebackground light exposure and the fluorescence light exposure may be setto be equal such that equivalent contributions from the pulsating lightcomponent are present in both exposures.

In some embodiments using a room light subtraction technique thatincludes taking a background light exposure, a background exposure imageintensity and/or a fluorescence exposure image intensity may be scaledbased on measurements of the pulsating room light intensity, in orderthat the scaled image intensities correspond to equivalent contributionsfrom the pulsating room light. After measuring the pulsating lightintensity and synthesizing a complete periodic cycle of the pulsatinglight intensity, as described herein, identification of the portion ofthe periodic cycle of the pulsating light intensity coinciding with afluorescence exposure and a background light exposure may be performedby repeating/extrapolating the periodic cycle as necessary to find theportion that coincided with the time spanned by each respectiveexposure. Calculating a fluorescence accumulated ambient light value,FL_(acc), corresponding to the accumulated ambient light intensityduring a fluorescence exposure may then be performed by calculating thearea under the curve marked by the portion of the periodic cycle ofpulsating light intensity for that exposure, and calculating abackground accumulated ambient light value, BG_(acc), corresponding tothe accumulated ambient light intensity during a background exposure maybe performed by calculating the area under the curve for the portion ofthe periodic pulsating light intensity coinciding with that exposure.Scaling the image intensity of a fluorescence image or a backgroundimage may then be performed based on a ratio of the respectiveaccumulated light values, FL_(acc) and BG_(acc), in order to normalizethe scaled images such that they reflect equivalent contributions ofaccumulated ambient light. Subsequent to scaling, the scaled backgroundimage may be subtracted from the scaled fluorescence image to yield acorrected fluorescence image that removes the ambient light signal andincludes correction for pulsatile ambient light contributions. In oneembodiment, one or the other of the fluorescence image or the backgroundimage is scaled by a factor of 1.

In embodiments where room light subtraction is not employed, thefluorescence exposure image intensity may be scaled based onmeasurements of the pulsating room light intensity, in order tofacilitate reducing image intensity artifacts resulting from thepulsating room light. For example, the scaling may be performed based ona ratio of measured intensities for successive fluorescence images.

To improve performance of ambient room light compensation methodsdescribed herein, a wavelength-dependent aperture (e.g., element 55 inFIG. 6A) may be used that includes a smaller central aperture thatpermits transmission of all visible and NIR light, and a surroundinglarger aperture that blocks visible light but permits transmission ofNIR light. Use of such a wavelength-dependent aperture allows a largerproportion of NIR signal to be collected relative to the visible lightsignal, which improves performance of the image signal subtraction forestimation and removal of the ambient room light component. Awavelength-dependent aperture may also feature a third, larger aperture,surrounding the other smaller apertures, that blocks both visible andNIR light. As an example, a wavelength-dependent aperture may comprise afilm aperture, wherein a film (e.g., a plastic or glass film) ofmaterial that blocks transmission of visible light but permitstransmission of NIR light has a central opening (e.g., a hole) thatpermits transmission of both visible and NIR light. Such a film aperturemay comprise material that blocks transmission of visible light throughreflection and/or material that blocks transmission of visible lightthrough absorption. As another example, a wavelength-dependent aperturemay comprise a dichroic aperture which is formed by masked thin-filmdeposition on a single substrate, wherein a thin-film that permitstransmission of visible and NIR light is deposited on a smaller centralaperture, and a second thin-film that blocks transmission of visiblelight but permits transmission of NIR light is deposited on asurrounding larger aperture. The respective aperture sizes of thesmaller central aperture and the surrounding larger aperture of awavelength-dependent aperture may be set in order to make the depth offield for visible light and for NIR light appear substantially similarwhen imaged by the imaging system. One or more wavelength-dependentfilters may be placed in different positions throughout the device,where rejection of the visible and passage of the NIR signal may beoptimized. For example, such a wavelength-dependent filter may bepositioned just before the lens 51. As another example, one or morewavelength-dependent filters may be placed in a pupil plane of theimaging lens.

It may be useful, e.g., to facilitate comparison of the fluorescencesignal of different regions, to display a target reticle around a regionwithin the imaged field of view, and to calculate and display thenormalized fluorescence intensity within that region. Normalization ofthe measured fluorescence intensity values may allow for meaningfulcomparison of multiple images and corresponding values. To correct forthe variation of measured fluorescence intensity with working distance(e.g., distance of the imaging system to the imaged anatomy), normalizedfluorescence intensity values may be based on a ratio between themeasured fluorescence intensity values and a reflected light valuewithin the target reticle region.

A numerical representation of the normalized fluorescence intensityvalue within the target reticle region may be displayed on or near theimage frame, to facilitate comparing values when aiming the targetreticle at different locations on the imaged anatomy. For example, thenumerical representation may be the mean value of the normalizedfluorescence intensity values for all of the image pixels in the targetreticle region.

Additionally or alternatively, a time history plot of the numericalrepresentation of the normalized fluorescence intensity value within thetarget reticle region may be displayed on or near the image frame, tofacilitate comparing values when aiming the target reticle at differentlocations on the imaged anatomy or at the same location over a series oftime points. Such a time history plot may further assist the user inassessing the fluorescence profile in the imaged tissue surface byscanning across the anatomy region of interest and viewing the relativenormalized fluorescence intensity profile plot.

FIG. 13A illustrates a diagram of a sample display output from anembodiment of the display method, wherein the target reticle 125 ispositioned over a region of no fluorescence intensity 122 on the imagedanatomy 120, and the numerical representation of the fluorescenceintensity 126 is displayed near the target reticle 125. FIG. 13Billustrates a diagram of another sample display output, wherein thetarget reticle 125 is positioned over a region of high relativenormalized fluorescence intensity 124, and showing a correspondingnumerical representation 126 of relatively high fluorescence intensity.FIG. 13C illustrates a diagram of another sample display output, whereinthe target reticle 125 is positioned over a region of moderate relativenormalized fluorescence intensity 124, and showing a correspondingnumerical representation 126 of relatively moderate fluorescenceintensity. FIG. 13D illustrates a diagram of a sample display output,wherein the target reticle 125 is positioned over a region of moderaterelative normalized fluorescence intensity 124, and showing a timehistory plot 128 of the numerical representation of normalizedfluorescence intensity that would be consistent with sequential imagingof regions of zero, high, and moderate relative normalized fluorescenceintensity. Alternatively or additionally to displaying the numericalrepresentation and/or historical plot on the target, a display regionassociated with the target reticle, e.g., on the device itself or someother display, may display this information.

FIG. 14 illustrates a recorded image of an anatomical fluorescenceimaging phantom, featuring an embodiment of a display method output thatdisplays normalized fluorescence intensity. In particular, a target 110is illuminated by excitation light in accordance with an embodiment anda target reticle 115 is positioned over a region of fluorescenceintensity 112. A numerical representation of the target reticle 115 isdisplayed in a region 116 associated with the target reticle 115 A timehistory plot 118 of the numerical representation of normalizedfluorescence intensity due to imaging of different positions of thereticle 115 may be displayed.

Normalization of the measured fluorescence intensity values mayadditionally or alternatively be performed on a pixel basis for anentire acquired fluorescence image or series of images, which mayfacilitate providing a consistent and/or smoothly varying imagebrightness, even when varying the working distance. To correct for thevariation of measured fluorescence intensity with working distance(e.g., distance of the imaging system to the imaged anatomy), normalizedfluorescence intensity values for each pixel in an acquired fluorescenceimage may be based on a ratio between the measured fluorescenceintensity value of that pixel and a reflected light value or componentof a reflected light value for the same pixel in an acquired reflectedlight image. In one embodiment, the reflected light image used for suchnormalization is a white light image formed from reflection of visiblewhite light illumination. For example, in embodiments in which a colorimage sensor is used to acquire the reflected light image, an overallluminance value, or a combination of one or more color channelintensities detected for each pixel from the color image sensor may beused.

Such a display method and/or technique for normalization of the measuredintensity values, as any one of those described herein, may be usefulfor a variety of fluorescence imaging systems, including an endoscopicor laparoscopic fluorescence imaging system, an open field fluorescenceimaging system, or a combination thereof. Such normalization and displayof the fluorescence intensity values can allow useful quantitativecomparisons of relative fluorescence intensity between image data fromvarious time points within an imaging session. Combined with appropriatestandardized fluorescent agent administration and imaging protocols, andstandardized calibration of imaging devices, such normalization anddisplay of the fluorescence intensity values can further allow usefulquantitative comparisons of relative fluorescence intensity betweenimage data from different imaging sessions.

EXAMPLES A Fluorescence Medical Imaging System for Acquisition of ImageData

In some embodiments, a system (also referred in some embodiments as adevice) for illumination and imaging of a subject may be used with or asa component of a medical imaging system such as, for example, afluorescence medical imaging system for acquiring fluorescence medicalimage data. An example of such a fluorescence medical imaging system isthe fluorescence imaging system 10 schematically illustrated in FIG. 1.In this embodiment, the fluorescence imaging system 10 is configured toacquire a time series of fluorescence signal intensity data (e.g.,images, video) capturing the transit of a fluorescence imaging agentthrough the tissue.

The fluorescence imaging system 10 (FIG. 1) comprises an illuminationsource 15 and illumination module 11 to illuminate the tissue of thesubject to induce fluorescence emission from a fluorescence imagingagent 17 in the tissue of the subject (e.g., in blood), an imagingmodule 13 configured to acquire the time series of fluorescence imagesfrom the fluorescence emission, and a processor assembly 16 configuredto utilize the acquired time series of fluorescence images (fluorescencesignal intensity data) according to the various embodiments describedherein.

In various embodiments, the illumination source 15 (FIG. 1) comprises,for example, a light source 200 (FIG. 15) comprising a fluorescenceexcitation source configured to generate an excitation light having asuitable intensity and a suitable wavelength for exciting thefluorescence imaging agent 17. The light source 200 in FIG. 15 comprisesa laser diode 202 (e.g., which may comprise, for example, one or morefiber-coupled diode lasers) configured to provide excitation light toexcite the fluorescence imaging agent 17 (not shown). Examples of othersources of the excitation light which may be used in various embodimentsinclude one or more LEDs, arc lamps, or other illuminant technologies ofsufficient intensity and appropriate wavelength to excite thefluorescence imaging agent 17 in the tissue (e.g., in blood). Forexample, excitation of the fluorescence imaging agent 17 in blood,wherein the fluorescence imaging agent 17 is a fluorescent dye with nearinfra-red excitation characteristics, may be performed using one or more793 nm, conduction-cooled, single bar, fiber-coupled laser diode modulesfrom DILAS Diode Laser Co, Germany.

In various embodiments, the light output from the light source 200 inFIG. 15 may be projected through an optical element (e.g., one or moreoptical elements) to shape and guide the output being used to illuminatethe tissue area of interest. The shaping optics may consist of one ormore lenses, light guides, and/or diffractive elements so as to ensure aflat field over substantially the entire field of view of the imagingmodule 13. In particular embodiments, the fluorescence excitation sourceis selected to emit at a wavelength close to the absorption maximum ofthe fluorescence imaging agent 17 (e.g., ICG). For example, referring tothe embodiment of the light source 200 in FIG. 15, the output 204 fromthe laser diode 202 is passed through one or more focusing lenses 206,and then through a homogenizing light pipe 208 such as, for example,light pipes commonly available from Newport Corporation, USA. Finally,the light is passed through an optical diffractive element 214 (e.g.,one or more optical diffusers) such as, for example, ground glassdiffractive elements also available from Newport Corporation, USA. Powerto the laser diode 202 itself is provided by, for example, ahigh-current laser driver such as those available from Lumina Power Inc.USA. The laser may optionally be operated in a pulsed mode during theimage acquisition process. In this embodiment, an optical sensor such asa solid state photodiode 212 is incorporated into the light source 200and samples the illumination intensity produced by the light source 200via scattered or diffuse reflections from the various optical elements.In various embodiments, additional illumination sources may be used toprovide guidance when aligning and positioning the module over the areaof interest. In various embodiments, at least one of the components oflight source 200 depicted in FIG. 15 may be components comprising theillumination source 15 and/or comprising the illumination module 11.

Referring back to FIG. 1, in various embodiments, the imaging module 13may be a component of, for example, the fluorescence imaging system 10configured to acquire the time series of fluorescence images (e.g.,video) from the fluorescence emission from the fluorescence imagingagent 17. Referring to FIG. 16, there is shown an exemplary embodimentof an imaging module 13 comprising a camera module 250. As is shown inFIG. 16, the camera module 250 acquires images of the fluorescenceemission 252 from the fluorescence imaging agent 17 in the tissue (e.g.,in blood) (not shown) by using a system of imaging optics (e.g., frontelement 254, rejection filter 256, dichroic 260 and rear element 262) tocollect and focus the fluorescence emission onto an image sensorassembly 264 comprising at least one 2D solid state image sensor. Arejection filter 256 may be, for example, a notch filter used to rejecta band of wavelengths corresponding to the excitation light. A dichroic260 may be, for example, a dichroic mirror used to selectively pass onesubset of the incoming light wavelength spectrum and redirect remainingwavelengths off of the optical path for rejection or towards a separateimage sensor. The solid state image sensor may be a charge coupleddevice (CCD), a CMOS sensor, a CID or similar 2D sensor technology. Thecharge that results from the optical signal transduced by the imagesensor assembly 264 is converted to an electrical video signal, whichincludes both digital and analog video signals, by the appropriateread-out and amplification electronics in the camera module 250.

According to some embodiments, excitation wavelength of about 800nm+/−10 nm and emission wavelengths of >820 nm are used along with NIRcompatible optics for ICG fluorescence imaging. A skilled person willappreciate that other excitation and emission wavelengths may be usedfor other imaging agents.

Referring back to FIG. 1, in various embodiments, the processor assembly16 comprises, for example,

-   -   a processor module (not shown) configured to perform various        processing operations, including executing instructions stored        on computer-readable medium, wherein the instructions cause one        or more of the systems described herein to execute the methods        and techniques described herein, and    -   a data storage module (not shown) to record and store the data        from the operations, as well as to store, in some embodiments,        instructions executable by the processor module to implement the        methods and techniques disclosed herein.

In various embodiments, the processor module comprises any computer orcomputing means such as, for example, a tablet, laptop, desktop,networked computer, or dedicated standalone microprocessor. Inputs aretaken, for example, from the image sensor 264 of the camera module 250shown in FIG. 16, from the solid state photodiode in the light source200 in FIG. 15, and from any external control hardware such as afootswitch or remote-control. Output is provided to the laser diodedriver, and optical alignment aids. In various embodiments, theprocessor assembly 16 (FIG. 1) may have a data storage module with thecapability to save the time series of input data (e.g., image data) to atangible non-transitory computer readable medium such as, for example,internal memory (e.g. a hard disk or flash memory), so as to enablerecording and processing of data. In various embodiments, the processormodule may have an internal clock to enable control of the variouselements and ensure correct timing of illumination and sensor shutters.In various other embodiments, the processor module may also provide userinput and graphical display of outputs. The fluorescence imaging systemmay optionally be configured with a video display (not shown) to displaythe images as they are being acquired or played back after recording, orfurther to visualize the data generated at various stages of the methodas was described above.

In operation, and with continuing reference to the exemplary embodimentsin FIGS. 1, 15 and 16, the subject is in a position for imaging wherethe anatomical area of interest of the subject is located beneath boththe illumination module 11 and the imaging module 13 such that asubstantially uniform field of illumination is produced acrosssubstantially the entire area of interest. In various embodiments, priorto the administration of the fluorescence imaging agent 17 to thesubject, an image may be acquired of the area of interest for thepurposes of background deduction. For example, in order to do this, theoperator of the fluorescence imaging system 10 in FIG. 1 may initiatethe acquisition of the time series of fluorescence images (e.g., video)by depressing a remote switch or foot-control, or via a keyboard (notshown) connected to the processor assembly 16. As a result, theillumination source 15 is turned on and the processor assembly 16 beginsrecording the fluorescence image data provided by the image acquisitionassembly 13. In lieu of the pulsed mode discussed above, it will beunderstood that, in some embodiments, the illumination source 15 cancomprise an emission source which is continuously on during the imageacquisition sequence. When operating in the pulsed mode of theembodiment, the image sensor 264 in the camera module 250 (FIG. 16) issynchronized to collect fluorescence emission following the laser pulseproduced by the diode laser 202. In the light source 200 (FIG. 15). Inthis way, maximum fluorescence emission intensity is recorded, andsignal-to-noise ratio is optimized. In this embodiment, the fluorescenceimaging agent 17 is administered to the subject and delivered to thearea of interest via arterial flow. Acquisition of the time series offluorescence images is initiated, for example, shortly afteradministration of the fluorescence imaging agent 17, and the time seriesof fluorescence images from substantially the entire area of interestare acquired throughout the ingress of the fluorescence imaging agent17. The fluorescence emission from the region of interest is collectedby the collection optics of the camera module 250. Residual ambient andreflected excitation light is attenuated by subsequent optical elements(e.g., optical element 256 in FIG. 16 which may be a filter) in thecamera module 250 so that the fluorescence emission can be acquired bythe image sensor assembly 264 with minimal interference by light fromother sources.

In various embodiments, the processor is in communication with theimaging system or is a component of the imaging system. The program codeor other computer-readable instructions, according to the variousembodiments, can be written and/or stored in any appropriate programminglanguage and delivered to the processor in various forms, including, forexample, but not limited to information permanently stored onnon-writeable storage media (e.g., read-only memory devices such as ROMsor CD-ROM disks), information alterably stored on writeable storagemedia (e.g., hard drives), information conveyed to the processor viatransitory mediums (e.g., signals), information conveyed to theprocessor through communication media, such as a local area network, apublic network such as the Internet, or any type of media suitable forstoring electronic instruction. In various embodiments, the tangiblenon-transitory computer readable medium comprises all computer-readablemedia. In some embodiments, computer-readable instructions forperforming one or more of the methods or techniques discussed herein maybe stored solely on non-transitory computer readable media.

In some embodiments, the illumination and imaging system may be acomponent of a medical imaging system such as the fluorescence medicalimaging system 10, which acquires medical image data. In embodimentswhere the illumination and imaging system is a component of the imagingsystem, such as the fluorescence imaging system described above, thelight source, illumination module, imaging module and the processor ofthe medical imaging system may function as the camera assembly and theprocessor of the illumination and imaging system. A skilled person willappreciate that imaging systems other than fluorescence imaging systemsmay be employed for use with illumination and/or imaging systems such asthose described herein, depending on the type of imaging beingperformed.

Example Imaging Agents for Use in Generating Image Data

According to some embodiments, in fluorescence medical imagingapplications, the imaging agent is a fluorescence imaging agent such as,for example, indocyanine green (ICG) dye. ICG, when administered to thesubject, binds with blood proteins and circulates with the blood in thetissue. The fluorescence imaging agent (e.g., ICG) may be administeredto the subject as a bolus injection (e.g., into a vein or an artery) ina concentration suitable for imaging such that the bolus circulates inthe vasculature and traverses the microvasculature. In other embodimentsin which multiple fluorescence imaging agents are used, such agents maybe administered simultaneously, e.g. in a single bolus, or sequentiallyin separate boluses. In some embodiments, the fluorescence imaging agentmay be administered by a catheter. In certain embodiments, thefluorescence imaging agent may be administered less than an hour inadvance of performing the measurement of signal intensity arising fromthe fluorescence imaging agent. For example, the fluorescence imagingagent may be administered to the subject less than 30 minutes in advanceof the measurement. In yet other embodiments, the fluorescence imagingagent may be administered at least 30 seconds in advance of performingthe measurement. In still other embodiments, the fluorescence imagingagent may be administered contemporaneously with performing themeasurement.

According to some embodiments, the fluorescence imaging agent may beadministered in various concentrations to achieve a desired circulatingconcentration in the blood. For example, in embodiments where thefluorescence imaging agent is ICG, it may be administered at aconcentration of about 2.5 mg/mL to achieve a circulating concentrationof about 5 μM to about 10 μM in blood. In various embodiments, the upperconcentration limit for the administration of the fluorescence imagingagent is the concentration at which the fluorescence imaging agentbecomes clinically toxic in circulating blood, and the lowerconcentration limit is the instrumental limit for acquiring the signalintensity data arising from the fluorescence imaging agent circulatingwith blood to detect the fluorescence imaging agent. In various otherembodiments, the upper concentration limit for the administration of thefluorescence imaging agent is the concentration at which thefluorescence imaging agent becomes self-quenching. For example, thecirculating concentration of ICG may range from about 2 μM to about 10mM. Thus, in one aspect, the method comprises the step of administrationof the imaging agent (e.g., a fluorescence imaging agent) to the subjectand acquisition of the signal intensity data (e.g., video) prior toprocessing the signal intensity data according to the variousembodiments. In another aspect, the method excludes any step ofadministering the imaging agent to the subject.

According to some embodiments, a suitable fluorescence imaging agent foruse in fluorescence imaging applications to generate fluorescence imagedata is an imaging agent which can circulate with the blood (e.g., afluorescence dye which can circulate with, for example, a component ofthe blood such as lipoproteins or serum plasma in the blood) and transitvasculature of the tissue large vessels and microvasculature), and fromwhich a signal intensity arises when the imaging agent is exposed toappropriate light energy (e.g., excitation light energy, or absorptionlight energy). In various embodiments, the fluorescence imaging agentcomprises a fluorescence dye, an analogue thereof, a derivative thereof,or a combination of these. A fluorescence dye includes any non-toxicfluorescence dye. In certain embodiments, the fluorescence dye optimallyemits fluorescence in the near-infrared spectrum. In certainembodiments, the fluorescence dye is or comprises a tricarbocyanine dye.In certain embodiments, the fluorescence dye is or comprises indocyaninegreen (ICG), methylene blue, or a combination thereof. In otherembodiments, the fluorescence dye is or comprises fluoresceinisothiocyanate, rhodamine, phycoerythrin, phycocyanin, allophycocyanin,o-phthaldehyde, fluorescamine, rose Bengal, trypan blue, fluoro-gold, ora combination thereof, excitable using excitation light wavelengthsappropriate to each dye. In some embodiments, an analogue or aderivative of the fluorescence dye may be used. For example, afluorescence dye analog or a derivative includes a fluorescence dye thathas been chemically modified, but still retains its ability to fluorescewhen exposed to light energy of an appropriate wavelength.

In various embodiments, the fluorescence imaging agent may be providedas a lyophilized powder, solid, or liquid. In certain embodiments, thefluorescence imaging agent may be provided in a vial (e.g., a sterilevial), which may permit reconstitution to a suitable concentration byadministering a sterile fluid with a sterile syringe. Reconstitution maybe performed using any appropriate carrier or diluent. For example, thefluorescence imaging agent may be reconstituted with an aqueous diluentimmediately before administration. In various embodiments, any diluentor carrier which will maintain the fluorescence imaging agent insolution may be used. As an example, ICG may be reconstituted withwater. In some embodiments, once the fluorescence imaging agent isreconstituted, it may be mixed with additional diluents and carriers. Insome embodiments, the fluorescence imaging agent may be conjugated toanother molecule, such as a protein, a peptide, an amino acid, asynthetic polymer, or a sugar, for example to enhance solubility,stability, imaging properties, or a combination thereof. Additionalbuffering agents may optionally be added including Tris, HCl, NaOH,phosphate buffer, and/or HEPES.

A person of skill in the art will appreciate that, although afluorescence imaging agent was described above in detail, other imagingagents may be used in connection with the systems, methods, andtechniques described herein, depending on the optical imaging modality.

In some variations, the fluorescence imaging agent used in combinationwith the methods, systems and kits described herein may be used forblood flow imaging, tissue perfusion imaging, lymphatic imaging, or acombination thereof, which may performed during an invasive surgicalprocedure, a minimally invasive surgical procedure, a non-invasivesurgical procedure, or a combination thereof. Examples of invasivesurgical procedure which may involve blood flow and tissue perfusioninclude a cardiac-related surgical procedure (e.g., CABG on pump or offpump) or a reconstructive surgical procedure. An example of anon-invasive or minimally invasive procedure includes wound (e.g.,chronic wound such as for example pressure ulcers) treatment and/ormanagement. In this regard, for example, a change in the wound overtime, such as a change in wound dimensions (e.g., diameter, area), or achange in tissue perfusion in the wound and/or around the peri-wound,may be tracked over time with the application of the methods andsystems. Examples of lymphatic imaging include identification of one ormore lymph nodes, lymph node drainage, lymphatic mapping, or acombination thereof. In some variations such lymphatic imaging mayrelate to the female reproductive system (e.g., uterus, cervix, vulva).

In variations relating to cardiac applications or any vascularapplications, the imaging agent(s) (e.g., ICG alone or in combinationwith another imaging agent) may be injected intravenously. For example,the imaging agent may be injected intravenously through the centralvenous line, bypass pump and/or cardioplegia line and/or othervasculature to flow and/or perfuse the coronary vasculature,microvasculature and/or grafts. ICG may be administered as a diluteICG/blood/saline solution down the grafted vessel or other vasculaturesuch that the final concentration of ICG in the coronary artery or othervasculature depending on application is approximately the same or loweras would result from injection of about 2.5 mg (i.e., 1 ml of 2.5 mg/ml)into the central line or the bypass pump. The ICG may be prepared bydissolving, for example, 25 mg of the solid in 10 ml sterile aqueoussolvent, which may be provided with the ICG by the manufacturer. Onemilliliter of the ICG solution may be mixed with 500 ml of sterilesaline (e.g., by injecting 1 ml of ICG into a 500 ml bag of saline).Thirty milliliters of the dilute ICG/saline solution may be added to 10ml of the subject's blood, which may be obtained in an aseptic mannerfrom the central arterial line or the bypass pump. ICG in blood binds toplasma proteins and facilitates preventing leakage out of the bloodvessels. Mixing of ICG with blood may be performed using standardsterile techniques within the sterile surgical field. Ten ml of theICG/saline/blood mixture may be administered for each graft. Rather thanadministering ICG by injection through the wall of the graft using aneedle, ICG may be administered by means of a syringe attached to the(open) proximal end of the graft. When the graft is harvested surgeonsroutinely attach an adaptor to the proximal end of the graft so thatthey can attach a saline filled syringe, seal off the distal end of thegraft and inject saline down the graft, pressurizing the graft and thusassessing the integrity of the conduit (with respect to leaks, sidebranches etc.) prior to performing the first anastomosis. In othervariations, the methods, dosages or a combination thereof as describedherein in connection with cardiac imaging may be used in any vascularand/or tissue perfusion imaging applications.

Lymphatic mapping is an important part of effective surgical staging forcancers that spread through the lymphatic system (e.g., breast, gastric,gynecological cancers). Excision of multiple nodes from a particularnode basin can lead to serious complications, including acute or chroniclymphedema, paresthesia, and/or seroma formation, when in fact, if thesentinel node is negative for metastasis, the surrounding nodes willmost likely also be negative. Identification of the tumor draining lymphnodes (LN) has become an important step for staging cancers that spreadthrough the lymphatic system in breast cancer surgery for example. LNmapping involves the use of dyes and/or radiotracers to identify the LNseither for biopsy or resection and subsequent pathological assessmentfor metastasis. The goal of lymphadenectomy at the time of surgicalstaging is to identify and remove the LNs that are at high risk forlocal spread of the cancer. Sentinel lymph node (SLN) mapping hasemerged as an effective surgical strategy in the treatment of breastcancer. It is generally based on the concept that metastasis (spread ofcancer to the axillary LNs), if present, should be located in the SLN,which is defined in the art as the first LN or group of nodes to whichcancer cells are most likely to spread from a primary tumor. If the SLNis negative for metastasis, then the surrounding secondary and tertiaryLN should also be negative. The primary benefit of SLN mapping is toreduce the number of subjects who receive traditional partial orcomplete lymphadenectomy and thus reduce the number of subjects whosuffer from the associated morbidities such as lymphedema andlymphocysts.

The current standard of care for SLN mapping involves injection of atracer that identifies the lymphatic drainage pathway from the primarytumor. The tracers used may be radioisotopes (e.g. Technetium-99 orTc-99m) for intraoperative localization with a gamma probe. Theradioactive tracer technique (known as scintigraphy) is limited tohospitals with access to radioisotopes require involvement of a nuclearphysician and does not provide real-time visual guidance. A colored dye,isosulfan blue, has also been used, however this dye cannot be seenthrough skin and fatty tissue. In addition, blue staining results intattooing of the breast lasting several months, skin necrosis can occurwith subdermal injections, and allergic reactions with rare anaphylaxishave also been reported. Severe anaphylactic reactions have occurredafter injection of isosulfan blue (approximately 2% of patients).Manifestations include respiratory distress, shock, angioedema,urticarial and pruritus. Reactions are more likely to occur in subjectswith a history of bronchial asthma, or subjects with allergies or drugreactions to triphenylmethane dyes. Isosulfan blue is known to interferewith measurements of oxygen saturation by pulse oximetry andmethemoglobin by gas analyzer. The use of isosulfan blue may result intransient or long-term (tattooing) blue coloration.

In contrast, fluorescence imaging in accordance with the variousembodiments for use in SLN visualization, mapping, facilitates directreal-time visual identification of a LN and/or the afferent lymphaticchannel intraoperatively, facilitates high-resolution optical guidancein real-time through skin and fatty tissue, visualization of blood flow,tissue perfusion or a combination thereof.

In some variations, visualization, classification or both of lymph nodesduring fluorescence imaging may be based on imaging of one or moreimaging agents, which may be further based on visualization and/orclassification with a gamma probe (e.g., Technetium Tc-99m is a clear,colorless aqueous solution and is typically injected into theperiareolar area as per standard care), another conventionally usedcolored imaging agent (isosulfan blue), and/or other assessment such as,for example, histology. The breast of a subject may be injected, forexample, twice with about 1% isosulfan blue (for comparison purposes)and twice with an ICG solution having a concentration of about 2.5mg/ml. The injection of isosulfan blue may precede the injection of ICGor vice versa. For example, using a TB syringe and a 30 G needle, thesubject under anesthesia may be injected with 0.4 ml (0.2 ml at eachsite) of isosulfan blue in the periareolar area of the breast. For theright breast, the subject may be injected at 12 and 9 o'clock positionsand for the left breast at 12 and 3 o'clock positions. The total dose ofintradermal injection of isosulfan blue into each breast may be about4.0 mg (0.4 ml of 1% solution: 10 mg/ml). in another exemplaryvariation, the subject may receive an ICG injection first followed byisosulfan blue for comparison). One 25 mg vial of ICG may bereconstituted with 10 ml sterile water for injection to yield a 2.5mg/ml solution immediately prior to ICG administration. Using a TBsyringe and a 30G needle, for example, the subject may be injected withabout 0.1 ml of ICG (0.05 ml at each site) in the periareolar area ofthe breast (for the right breast, the injection may be performed at 12and 9 o'clock positions and for the left breast at 12 and 3 o'clockpositions). The total dose of intradermal injection of ICG into eachbreast may be about 0.25 mg (0.1 ml of 2.5 mg/ml solution) per breast,ICG may be injected, for example, at a rate of 5 to 10 seconds perinjection. When ICG is injected intradermally, the protein bindingproperties of ICG cause it to be rapidly taken up by the lymph and movedthrough the conducting vessels to the LN. In some variations, the ICGmay be provided in the form of a sterile lyophilized powder containing25 mg ICG with no more than 5% sodium iodide. The ICG may be packagedwith aqueous solvent consisting of sterile water for injection, which isused to reconstitute the ICG. In some variations the ICG dose (mg) inbreast cancer sentinel lymphatic mapping may range from about 0.5 mg toabout 10 mg depending on the route of administration. In somevariations, the ICG does may be about 0.6 mg to about 0.75 mg, about0.75 mg to about 5 mg, about 5 mg to about 10 mg. The route ofadministration may be for example subdermal, intradermal (e.g., into theperiareolar region), subareolar, skin overlaying the tumor, intradermalin the areola closest to tumor, subdermal into areola, intradermal abovethe tumor, periareolar over the whole breast, or a combination thereof.The NIR fluorescent positive LNs (e.g., using ICG) may be represented asa black and white NIR fluorescence image(s) for example and/or as a fullor partial color (white light) image, full or partial desaturated whitelight image, an enhanced colored image, an overlay (e.g., fluorescencewith any other image), a composite image (e.g., fluorescenceincorporated into another image) which may have various colors, variouslevels of desaturation or various ranges of a color tohighlight/visualize certain features of interest. Processing of theimages may be further performed for further visualization and/or otheranalysis (e.g., quantification). The lymph nodes and lymphatic vesselsmay be visualized (e.g., intraoperatively, in real time) usingfluorescence imaging systems and methods according to the variousembodiments for ICG and SLNs alone or in combination with a gamma probe(Tc-99m) according to American Society of Breast Surgeons (ASBrS)practice guidelines for SLN biopsy in breast cancer patients.Fluorescence imaging for LNs may begin from the site of injection bytracing the lymphatic channels leading to the LNs in the axilla. Oncethe visual images of LNs are identified, LN mapping and identificationof LNs may be done through incised skin, LN mapping may be performeduntil ICG visualized nodes are identified. For comparison, mapping withisosulfan blue may be performed until ‘blue’ nodes are identified. LNsidentified with ICG alone or in combination with another imagingtechnique (e,g., isosulfan blue, and/or Tc-99m) may be labeled to beexcised. Subject may have various stages of breast cancer (e.g., IA, IB,IIA).

In some variations, such as for example, in gynecological cancers (e.g.,uterine, endometrial, vulvar and cervical malignancies), ICG may beadministered interstitially for the visualization of lymph nodes,lymphatic channels, or a combination thereof. When injectedinterstitially, the protein binding properties of ICG cause it to berapidly taken up by the lymph and moved through the conducting vesselsto the SLN, ICG may be provided for injection in the form of a sterilelyophilized powder containing 25 mg ICG (e.g., 25 mg/vial) with no morethan 5.0% sodium iodide. ICG may be then reconstituted with commerciallyavailable water (sterile) for injection prior to use. According to anembodiment, a vial containing 25 mg ICG may be reconstituted in 20 ml ofwater for injection, resulting in a 1.25 mg/ml solution. A total of 4 mlof this 1.25 mg/ml solution is to be injected into a subject (4×1 mlinjections) for a total dose of ICG of 5 mg per subject. The cervix mayalso be injected four (4) times with a 1 ml solution of 1% isosulfanblue 10 mg/ml (for comparison purposes) for a total dose of 40 mg. Theinjection may be performed while the subject is under anesthesia in theoperating room. In some variations the ICG dose (mg) in gynecologicalcancer sentinel lymph node detection and/or mapping may range from about0.1 mg to about 5 mg depending on the route of administration. In somevariations, the ICG does may be about 0.1 mg to about 0.75 mg, about0.75 mg to about 1.5 mg, about 1.5 mg to about 2.5 mg, about 2.5 mg toabout 5 mg. The route of administration may be for example cervicalinjection, vulva peritumoral injection, hysteroscopic endometrialinjection, or a combination thereof. In order to minimize the spillageof isosulfan blue or ICG interfering with the mapping procedure when LNsare to be excised, mapping may be performed on a hemi-pelvis, andmapping with both isosulfan blue and ICG may be performed prior to theexcision of any LNs. LN mapping for Clinical Stage I endometrial cancermay be performed according to the NCCN Guidelines for Uterine Neoplasms,SLN Algorithm for Surgical Staging of Endometrial Cancer; and SLNmapping for Clinical Stage I cervical cancer may be performed accordingto the NCCN Guidelines for Cervical Neoplasms, Surgical/SLN MappingAlgorithm for Early-Stage Cervical Cancer. Identification of LNs maythus be based on ICG fluorescence imaging alone or in combination orco-administration with for a colorimetric dye (isosulfan blue) and/orradiotracer.

Visualization of lymph nodes may be qualitative and/or quantitative.Such visualization may comprise, for example, lymph node detection,detection rate, anatomic distribution of lymph nodes. Visualization oflymph nodes according to the various embodiments may be used alone or incombination with other variables (e.g., vital signs, height, weight,demographics, surgical predictive factors, relevant medical history andunderlying conditions, histological visualization and/or assessment,Tc-99m visualization and/or assessment, concomitant medications).Follow-up visits may occur on the date of discharge, and subsequentdates (e.g., one month).

Lymph fluid comprises high levels of protein, thus ICG can bind toendogenous proteins when entering the lymphatic system. Fluorescenceimaging (e.g., ICG imaging) for lymphatic mapping when used inaccordance with the methods and systems described herein offers thefollowing example advantages: high-signal to background ratio (or tumorto background ratio) as NIR does not generate significantautofluorescence, real-time visualization feature for lymphatic mapping,tissue definition (i.e., structural visualization), rapid excretion andelimination after entering the vascular system, and avoidance ofnon-ionizing radiation. Furthermore, NIR imaging has superior tissuepenetration (approximately 5 to 10 millimeters of tissue) to that ofvisible light (1 to 3 mm of tissue). The use of ICG for example alsofacilitates visualization through the peritoneum overlying thepara-aortic nodes. Although tissue fluorescence can be observed with NIRlight for extended periods, it cannot be seen with visible light andconsequently does not impact pathologic evaluation or processing of theLN. Also, fluorescence is easier to detect intra-operatively than bluestaining (isosulfan blue) of lymph nodes. In other variations, themethods, dosages or a combination thereof as described herein inconnection with lymphatic imaging may be used in any vascular and/ortissue perfusion imaging applications.

Tissue perfusion relates to the microcirculatory flow of blood per unittissue volume in which oxygen and nutrients are provided to and waste isremoved from the capillary bed of the tissue being perfused. Tissueperfusion is a phenomenon related to but also distinct from blood flowin vessels. Quantified blood flow through blood vessels may be expressedin terms that define flow (i.e., volume/time), or that define speed(i.e., distance/time). Tissue blood perfusion defines movement of bloodthrough micro-vasculature, such as arterioles, capillaries, or venules,within a tissue volume. Quantified tissue blood perfusion may beexpressed in terms of blood flow through tissue volume, namely, that ofblood volume/time/tissue volume (or tissue mass). Perfusion isassociated with nutritive blood vessels (e.g., micro-vessels known ascapillaries) that comprise the vessels associated with exchange ofmetabolites between blood and tissue, rather than larger-diameternon-nutritive vessels. In some embodiments, quantification of a targettissue may include calculating or determining a parameter or an amountrelated to the target tissue, such as a rate, size volume, time,distance/time, and/or volume/time, and/or an amount of change as itrelates to any one or more of the preceding parameters or amounts.However, compared to blood movement through the larger diameter bloodvessels, blood movement through individual capillaries can be highlyerratic, principally due to vasomotion, wherein spontaneous oscillationin blood vessel tone manifests as pulsation in erythrocyte movement.

By way of summation and review, one or more embodiments may accommodatevaried working distances while providing a flat illumination field andmatching an illumination field to a target imaging field, thus allowingaccurate quantitative imaging applications. An imaging element thatfocuses light from a target onto a sensor may be moved in synchrony withsteering of the illumination field. Additionally or alternatively, adrape may be used that insures a close fit between a drape lens and awindow frame of the device. Additionally or alternatively, one or moreembodiments may allow ambient light to be subtracted from light to beimaged using a single sensor and controlled timing of illumination andexposure or detection. Additionally or alternatively, one or moreembodiments may allow the display of a normalized fluorescence intensitymeasured within a target reticle region of an image frame.

In contrast, when illumination and imaging devices do not conformillumination to the target imaging field of view or provide a flat,i.e., even or substantially uniform, illumination field, illuminationand image quality may suffer. An uneven illumination field can causedistracting and inaccurate imaging artifacts, especially for hand heldimaging devices and when used at varied working distances, while excesslight outside the imaging field of view reduces device efficiency andcan distract the user when positioning the device.

The methods and processes described herein may be performed by code orinstructions to be executed by a computer, processor, manager, orcontroller, or in hardware or other circuitry. Because the algorithmsthat form the basis of the methods (or operations of the computer,processor, or controller) are described in detail, the code orinstructions for implementing the operations of the method embodimentsmay transform the computer, processor, or controller into aspecial-purpose processor for performing the methods described herein.

Also, another embodiment may include a computer-readable medium, e.g., anon-transitory computer-readable medium, for storing the code orinstructions described above. The computer-readable medium may be avolatile or non-volatile memory or other storage device, which may beremovably or fixedly coupled to the computer, processor, or controllerwhich is to execute the code or instructions for performing the methodembodiments described herein.

One or more embodiments are directed to an illumination module for usein an imaging system having an imaging field of view for imaging atarget, the illumination module including a first illumination port tooutput a first light beam having a first illumination distribution atthe target to illuminate the target and a second illumination part tooutput a second light beam having a second illumination distribution atthe target to illuminate the target. The second illuminationdistribution may be substantially similar to the first illuminationdistribution at the target, the second illumination port being spacedapart from the first illumination port, the first and secondillumination distributions being simultaneously provided to the targetand overlapping at the target, wherein the illumination from the firstand second ports is matched to a same aspect ratio and field of viewcoverage as the imaging field of view.

Light from the first and second illumination ports may respectivelyoverlap to provide uniform illumination over a target field of view.

The illumination module may include a steering driver to simultaneouslysteer the first and second illumination ports through different fieldsof view.

Each of the first and second illumination ports may include a lensmodule having at least one fixed lens, a steerable housing, and at leastone lens mounted in the steerable housing, the steerable housing beingin communication with the steering driver.

The illumination module may include an enclosure, the enclosure housingthe first and second illumination ports and the steering driver.

The enclosure may be a hand held enclosure and may include a controlsurface including activation devices to control the steering driver.

Each of the first and second illumination distributions may be arectangular illumination distribution.

Each of the first and second illumination ports may include a lensmodule having two pairs of cylindrical lenses.

The first and second illumination ports may be symmetrically offset froma long dimension midline of the rectangular illumination distribution.

One or more embodiments are directed to an imaging device having animaging field of view, the imaging device including a first illuminationport to output first light having a first illumination distribution at atarget to illuminate the target, a second illumination port to outputsecond light having a second illumination distribution at the target toilluminate the target, the second illumination distribution beingsubstantially similar to the first illumination distribution at thetarget, the second illumination port being spaced apart from the firstillumination port, the first and second illumination distributions beingsimultaneously provided to the target and overlapping at the target,wherein the illumination from the first and second ports is matched to asame aspect ratio and field of view coverage as the imaging field ofview, and a sensor to detect light from the target.

The imaging device may include an enclosure, the enclosure housing thefirst and second illumination ports, and the sensor.

The imaging device may include a steering driver to simultaneously steerthe first and second illumination ports through different fields ofview.

The imaging device may include an imaging element to focus light ontothe sensor, wherein the steering driver is to move the imaging elementin synchrony with steering of the first and second illumination ports.

The steering driver may be in the enclosure and the enclosure mayinclude a control surface including activation devices to control thesteering driver.

The enclosure may have a hand held enclosure having a form factor thatallows a single hand to control the control surface and illumination ofthe target from multiple orientations.

The imaging device may include an illumination source to output light tothe first and second illumination ports, the illumination source beingoutside the enclosure.

The illumination source may output visible light and/or excitation lightto the first and second illumination ports.

The sensor may be a single sensor that is to detect light from thetarget resulting from illumination by visible light and excitationlight.

The imaging device may include a wavelength-dependent aperture upstreamof the sensor, the wavelength-dependent aperture to block visible lightoutside a central region.

The imaging device may include a video processor box, the videoprocessor box being outside the enclosure.

The illumination source may be integral with the video processor box.

One or more embodiments are directed to a method of examining a target,the method including simultaneously illuminating the target with a firstlight output having a first illumination distribution at the target andwith a second light output having a second illumination distribution atthe target, the second illumination distribution being substantiallysimilar to the first illumination distribution, the first and secondillumination distributions overlapping at the target, wherein theillumination on the target is matched to the same aspect ratio and fieldof view coverage as an imaging field of view.

The method may include simultaneously steering the first and secondlight outputs through different fields of view,

The method may include receiving light from the target and focusinglight onto a sensor using an imaging element, the imaging element beingmoved in synchrony with simultaneous steering of the first and secondlight outputs.

One or more embodiments are directed to a drape for use with an imagingdevice, the drape including a barrier material enveloping the imagingdevice, a drape window frame defining an opening in the barriermaterial, a drape lens in the opening in the barrier material, and aninterface integral with the drape window frame to secure the drape lensto a window frame of the imaging device.

The drape may be insertable into the window frame of the imaging device.

The interface may include two clamps integrated symmetrically onrespective opposing sides of the drape window frame.

The two clamps are on a top and a bottom of the drape window frame.

One or more embodiments are directed to a processor to image a target,the processor to, within a period, turn on an excitation light source togenerate an excitation pulse to illuminate the target, turn on a whitelight source to generate a white pulse to illuminate the target suchthat the white pulse does not overlap the excitation pulse and the whitepulse is generated at least twice within the period, expose an imagesensor for a fluorescent exposure time during the excitation pulse,expose the image sensor for a visible exposure time during at least onewhite pulse, detect outputs from the image sensor, compensate forambient light, and output a resultant image.

To compensate for ambient light, the processor may expose a first set ofsensor pixel rows of the image sensor for a fraction of the fluorescentexposure time for a first set of sensor pixel rows; and expose a secondset of sensor pixel rows of the image sensor for all of the fluorescentexposure time, the first and second sets to detect at least onedifferent color from the other.

The fraction may be ½.

The processor may determine the fluorescent signal F using the followingequation:

F=2*Exp2−Exp1,

where Exp1 is a signal output during the fraction of fluorescentexposure time and Exp2 is a signal output during all of the fluorescentexposure time.

The fraction of the exposure time may equal a width of the excitationpulse.

The visible exposure time may be longer than a width of the at least onewhite pulse.

The visible exposure time may be for one white pulse within the period.

The visible exposure time may be for two white pulses within the period.

To compensate for ambient light, the processor may expose the imagesensor for a background exposure time when target is not illuminated atleast once within the period.

One or more embodiments are directed a method for imaging a target,within a period, the method including generating an excitation pulse toilluminate the target, generating a white pulse to illuminate the targetsuch that the white pulse does not overlap the excitation pulse and thewhite pulse is generated at least twice within the period, exposing animage sensor for a fluorescent exposure time during the excitationpulse, exposing the image sensor for a visible exposure time during atleast one white pulse, detecting outputs from the image sensor,compensating for ambient light, and outputting a resultant image.

Compensating for ambient light may include exposing a first set ofsensor pixel rows of the image sensor for a fraction of the fluorescentexposure time and exposing a second set of sensor pixel rows of theimage sensor for all of the fluorescent exposure time, the first andsecond sets to detect at least one different color from the other.

Compensating for ambient light may include exposing the image sensor fora background exposure time when target is not illuminated at least oncewithin the period.

Generating the excitation pulse may include providing uniform,anamorphic illumination to the target.

Providing uniform, anamorphic illumination to the target includesoverlapping illumination from at least two illumination ports.

One or more embodiments are directed to a method of displayingfluorescence intensity in an image, the method including displaying atarget reticle covering a region of the image, calculating a normalizedfluorescence intensity within the target reticle, and displaying thenormalized fluorescence intensity in a display region associated withthe target.

The display region may be projected onto the target.

The normalized fluorescence intensity may include a single numericalvalue and/or a historical plot of normalized fluorescence intensities.

One or more embodiments are directed to a kit, including an illuminationmodule including at least two illumination ports spaced apart from oneanother, first and second illumination distributions to beingsimultaneously provided to a target and to overlap at the target, and animaging module including a sensor to detect light from the target.

The kit may include an enclosure to enclose the illumination module andthe imaging module.

One or more embodiments are directed to a fluorescence imaging agent foruse in the imaging device and methods as described herein. In one ormore embodiments, the use may comprise blood flow imaging, tissueperfusion imaging, lymphatic imaging, or a combination thereof, whichmay occur during an invasive surgical procedure, a minimally invasivesurgical procedure, a non-invasive surgical procedure, or a combinationthereof. The fluorescence agent may be included in the kit describedherein.

In one or more embodiments, the invasive surgical procedure may comprisea cardiac-related surgical procedure or a reconstructive surgicalprocedure. The cardiac-related surgical procedure may comprise a cardiaccoronary artery bypass graft (CABG) procedure which may be on pumpand/or off pump.

In one or more embodiments, the minimally invasive or the non-invasivesurgical procedure may comprise a wound care procedure.

In one or more embodiments, the lymphatic imaging may compriseidentification of a lymph node, lymph node drainage, lymphatic mapping,or a combination thereof. The lymphatic imaging may relate to the femalereproductive system.

Example embodiments have been disclosed herein, and although specificterms are employed, they are used and are to be interpreted in a genericand descriptive sense only and not for purpose of limitation. In someinstances, as would be apparent to one of ordinary skill in the art asof the filing of the present application, features, characteristics,and/or elements described in connection with a particular embodiment maybe used singly or in combination with features, characteristics, and/orelements described in connection with other embodiments unless otherwisespecifically indicated. Accordingly, it will be understood by those ofskill in the art that various changes in form and details may be madewithout departing from the spirit and scope of the present invention asset forth in the following.

While the present disclosure has been illustrated and described inconnection with various embodiments shown and described in detail, it isnot intended to be limited to the details shown, since variousmodifications and structural changes may be made without departing inany way from the scope of the present disclosure. Various modificationsof form, arrangement of components, steps, details and order ofoperations of the embodiments illustrated, as well as other embodimentsof the disclosure may be made without departing in any way from thescope of the present disclosure, and will be apparent to a person ofskill in the art upon reference to this description. It is thereforecontemplated that the appended claims will cover such modifications andembodiments as they fall within the true scope of the disclosure. Forthe purpose of clarity and a concise description, features are describedherein as part of the same or separate embodiments, however, it will beappreciated that the scope of the disclosure includes embodiments havingcombinations of all or some of the features described. For the terms“for example” and “such as,” and grammatical equivalences thereof, thephrase “and without limitation” is understood to follow unlessexplicitly stated otherwise. As used herein, the singular forms “a”,“an”, and “the” include plural referents unless the context clearlydictates otherwise.

What is claimed is:
 1. An imaging device having an imaging field ofview, the imaging device comprising: at least one illumination portconfigured to output light for illuminating a target; an imaging sensorto detect light traveling along an optical path to the imaging sensor;and a first movable window positioned upstream of the sensor withrespect to a direction of travel of light along the optical path,wherein the first movable window is configured to move into the opticalpath in a deployed position for modifying light received from thetarget.
 2. The imaging device of claim 1, wherein the first movablewindow is configured to rotate into the optical path in a deployedposition.
 3. The imaging device of claim 1, wherein the first movablewindow is configured to translate into the optical path in a deployedposition.
 4. The imaging device of claim 1, wherein the first movablewindow extends perpendicularly to an optical axis in the deployedposition.
 5. The imaging device of claim 1, wherein the first movablewindow is configured to pivot into the optical path in a deployedposition.
 6. The imaging device of claim 5, wherein the first movablewindow is configured to pivot about a first pivot axis extendingperpendicularly to an optical axis.
 7. The imaging device of claim 1,wherein the first movable window comprises a filter.
 8. The imagingdevice of claim 7, wherein the filter is configured to filter outvisible light.
 9. The imaging device of claim 1, comprising a secondmovable window positioned upstream of the imaging sensor with respect tothe direction of travel of light along the optical path, wherein thesecond movable window is configured to move into the optical path in adeployed position for modifying light received from the target.
 10. Theimaging device of claim 9, wherein the second movable window isconfigured to pivot about a second pivot axis extending perpendicularlyto an optical axis.
 11. The imaging device of claim 10, wherein thefirst movable window is configured to pivot about a first pivot axisextending perpendicularly to the optical axis and the first pivot axisand the second pivot axis are coplanar with a plane extendingperpendicularly to the optical axis.
 12. The imaging device of claim 9,wherein the first movable window and the second movable window arecoupled to a linkage that is configured to simultaneously move the firstand second pivoting windows.
 13. The imaging device of claim 9, wherein,when the first movable window is in the deployed position, the secondmovable window is moved out of the optical path in a stowed position.14. The imaging device of claim 1, wherein the image sensor istranslatable with respect to the first movable window.
 15. The imagingdevice of claim 14, wherein the first movable window extendsperpendicularly to an optical axis in the deployed position and theimage sensor is translatable along the optical axis.
 16. The imagingdevice of claim 1, comprising a first illumination port and a secondillumination port, wherein the first illumination port is configured togenerate a first illumination distribution at the target, the secondillumination port is configured to generate a second illuminationdistribution at the target, the second illumination port is spaced apartfrom the first illumination port, the first and second illuminationdistributions are simultaneously provided to the target and overlap atthe target, and the illumination from the first and second ports ismatched to a same aspect ratio and field of view coverage as the imagingfield of view.
 17. The imaging device of claim 16, wherein the first andsecond illumination ports are fixed with respect to each other.
 18. Theimaging device of claim 1, wherein the at least one illumination port isconfigured to output at least one of visible light and excitation light.19. The imaging device of claim 18, wherein the image sensor is a singlesensor that is configured to detect light from the target resulting fromillumination by visible light and excitation light.
 20. The imagingdevice of claim 19, comprising a wavelength-dependent aperture upstreamof the image sensor, wherein the wavelength-dependent aperture isconfigured to block visible light outside a central region.
 21. Theimaging device of claim 1, comprising one or more sensors for sensing anamount of light incident on the device.
 22. The imaging device of claim21, comprising a control system configured to adjust at least one imageacquisition parameter based on output from the one or more sensors. 23.The imaging device of claim 22, wherein the at least one imageacquisition parameter comprises an exposure duration, excitationillumination duration, excitation illumination power, or imaging sensorgain.
 24. The imaging device of claim 21, wherein at least one of theone or more sensors is configured to sense visible light and nearinfrared light.
 25. The imaging device of claim 21, wherein at least oneof the one or more sensors is configured to sense near infrared light.26. The imaging device of claim 1, comprising one or more drape sensorsconfigured to detect a drape mounted to the device.
 27. The imagingdevice of claim 26, comprising one or more light emitters for emittinglight for detection by the one or more drape sensors.
 28. The imagingdevice of claim 27, wherein the one or more drape sensors are configuredto detect light emitted from the one or more light emitters afterreflection of the emitted light off of one or more reflectors on thedrape.
 29. The imaging device of claim 28, wherein the one or morereflectors comprise a prism.
 30. A method for imaging a target, themethod comprising: illuminating the target with an illuminator of animaging device; receiving light from the target at an imaging sensor ofthe imaging device in a first imaging mode, wherein at least some of thelight received at the imaging sensor in the first imaging mode compriseswavelengths in a first band; switching to a second imaging mode; andwhile in the second imaging mode: blocking light of wavelengths outsideof a second band received from the target from reaching the imagingsensor using a first movable filter of the imaging device, wherein atleast some of the blocked light comprises wavelengths in the first band,and receiving light of wavelengths within the second band received fromthe target on the imaging sensor.
 31. The method of claim 30, whereinthe second band comprises near infrared wavelengths.
 32. The method ofclaim 30, wherein the first band comprises visible light wavelengths.33. The method of claim 30, comprising, while in the second imagingmode, sensing light levels at one or more light level sensors of theimaging device and adjusting one or more of image sensor signal gain,illumination pulse duration, image sensor exposure, and illuminationpower based on output of the one or more light level sensors.
 34. Themethod of claim 33, comprising, while in the first imaging mode, sensinglight levels at one or more light level sensors of the imaging deviceand adjusting one or more of image sensor signal gain, illuminationpulse duration, image sensor exposure, and illumination power based onoutput of the one or more light level sensors.
 35. The method of claim30, wherein switching to the second imaging mode comprises moving thefirst movable filter into an optical path along which light from thetarget travels to the imaging sensor.
 36. The method of claim 35,wherein switching to the second imaging mode comprises moving a clearwindow out of the optical path.
 37. The method of claim 35, whereinswitching to the second imaging mode comprises moving a second movablefilter out of the optical path.
 38. The method of claim 30, wherein thefirst imaging mode is switched to the second imaging mode in response toa user request.
 39. The method of claim 38, wherein the user requestcomprises a user input o the imaging device.
 40. The method of claim 30,comprising: while in the second imaging mode, receiving a request fromthe user to switch to the first imaging mode; and in response toreceiving the request from the user to switch to the first imaging mode,moving the movable filter out of the optical path.
 41. The method ofclaim 40, comprising: while in the second imaging mode, sensing lightlevels at one or more light level sensors of the imaging device andadjusting one or more of image sensor signal gain, illumination pulseduration, image sensor exposure, and illumination power based on outputof the one or more light level sensors; and in response to receiving therequest from the user to switch to the first imaging mode, ceasing toadjust one or more of image sensor signal gain, illumination pulseduration, image sensor exposure, and illumination power based on outputof the one or more light level sensors.
 42. The method of claim 30,comprising detecting an object at least partially blocking anillumination beam of the illuminator, and in response to detecting theobject, adjusting an illumination power of the illuminator.
 43. A systemfor imaging a target, the system comprising: one or more processors;memory; and one or more programs, wherein the one or more programs arestored in the memory and configured to be executed by the one or moreprocessors, the one or more programs including instructions for, withina period: activating an excitation light source to generate anexcitation pulse to illuminate the target; receiving an ambient lightintensity signal from a sensor during a portion of the period in whichthe excitation light source is not activated; exposing an image sensorfor a fluorescent exposure time during the excitation pulse; receivingoutputs from the image sensor; compensating for ambient light based onthe ambient light intensity signal; and storing a resultant image in thememory.
 44. The system of claim 43, wherein the one or more programsinclude instructions for, within the period: activating a white lightsource to generate a white light pulse to illuminate the target such atthe white light pulse does not overlap the excitation pulse; andexposing the image sensor for a visible exposure time during at leastone white light pulse.
 45. The system of claim 43, wherein the one ormore programs include instructions for exposing the image sensor for abackground exposure time when the target is not illuminated.
 46. Thesystem of claim 43, wherein the one or more programs includeinstructions for detecting a periodic frequency of the ambient lightintensity.
 47. The system of claim 46, wherein compensating for ambientlight comprises: setting an image acquisition frame rate equal to amultiple or a factor of the periodic frequency prior to exposing theimage sensor for the background exposure time and prior to exposing theimage sensor for the fluorescent exposure time during the excitationpulse; and subtracting image sensor output received for the backgroundexposure time from the image sensor output received for the fluorescenceexposure time to form the resultant image.
 48. The system of claim 46,wherein compensating for ambient light comprises: synthesizing orextracting, from one or more received ambient light intensity signals, acomplete periodic cycle of ambient light intensity having the detectedperiodic frequency; extending the ambient light intensity periodic cycleto a time period corresponding to the fluorescence exposure time;calculating a first accumulated ambient light value corresponding to anarea under the curve of ambient light intensity during a backgroundexposure time; calculating a second accumulated ambient light valuecorresponding to an area under the curve of the ambient light intensityduring the fluorescence exposure time; scaling the received image sensoroutput for the background exposure time and the received image sensoroutput for the fluorescence exposure time based on a ratio of the firsand second accumulated ambient light values; and subtracting the scaledimage sensor output for the background exposure time from the scaledimage sensor output for the fluorescence exposure time to form theresultant image.
 49. The system of claim 48, wherein the one or moreprograms include instructions for receiving an ambient light intensitysignal from the sensor during the background exposure time.
 50. Thesystem of claim 48, wherein the one or more programs includeinstructions for extending the ambient light intensity periodic cycle tothe time period corresponding to the fluorescence exposure time.
 51. Amethod for imaging a target, the method comprising: at a system havingone or more processors and memory: activating an excitation light sourceto generate an excitation pulse to illuminate the target; receiving anambient light intensity signal from a sensor during a portion of theperiod in which the excitation light source is not activated; exposingan image sensor for a fluorescent exposure time during the excitationpulse; receiving outputs from the image sensor; compensating for ambientlight based on the ambient light intensity signal; and storing aresultant image in the memory.